2023-02-11 00:53:54,782 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-11 00:53:55,010 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6 2023-02-11 00:53:55,027 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-11 00:53:55,028 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6, deleteOnExit=true 2023-02-11 00:53:55,028 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-11 00:53:55,028 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/test.cache.data in system properties and HBase conf 2023-02-11 00:53:55,029 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/hadoop.tmp.dir in system properties and HBase conf 2023-02-11 00:53:55,029 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/hadoop.log.dir in system properties and HBase conf 2023-02-11 00:53:55,030 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-11 00:53:55,030 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-11 00:53:55,030 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-11 00:53:55,165 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-11 00:53:55,536 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-11 00:53:55,540 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-11 00:53:55,540 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-11 00:53:55,540 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-11 00:53:55,541 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 00:53:55,541 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-11 00:53:55,541 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-11 00:53:55,541 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 00:53:55,542 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 00:53:55,542 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-11 00:53:55,542 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/nfs.dump.dir in system properties and HBase conf 2023-02-11 00:53:55,542 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/java.io.tmpdir in system properties and HBase conf 2023-02-11 00:53:55,543 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 00:53:55,543 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-11 00:53:55,544 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-11 00:53:56,039 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 00:53:56,043 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 00:53:56,316 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-11 00:53:56,475 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-11 00:53:56,492 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 00:53:56,532 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 00:53:56,573 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/java.io.tmpdir/Jetty_localhost_40361_hdfs____.mym1b8/webapp 2023-02-11 00:53:56,727 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40361 2023-02-11 00:53:56,738 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 00:53:56,738 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 00:53:57,137 WARN [Listener at localhost/46775] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 00:53:57,239 WARN [Listener at localhost/46775] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 00:53:57,257 WARN [Listener at localhost/46775] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 00:53:57,262 INFO [Listener at localhost/46775] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 00:53:57,266 INFO [Listener at localhost/46775] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/java.io.tmpdir/Jetty_localhost_37683_datanode____.73c98/webapp 2023-02-11 00:53:57,360 INFO [Listener at localhost/46775] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37683 2023-02-11 00:53:57,674 WARN [Listener at localhost/44865] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 00:53:57,687 WARN [Listener at localhost/44865] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 00:53:57,690 WARN [Listener at localhost/44865] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 00:53:57,692 INFO [Listener at localhost/44865] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 00:53:57,699 INFO [Listener at localhost/44865] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/java.io.tmpdir/Jetty_localhost_36027_datanode____vgzk7t/webapp 2023-02-11 00:53:57,803 INFO [Listener at localhost/44865] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36027 2023-02-11 00:53:57,813 WARN [Listener at localhost/35403] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 00:53:57,830 WARN [Listener at localhost/35403] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 00:53:57,834 WARN [Listener at localhost/35403] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 00:53:57,836 INFO [Listener at localhost/35403] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 00:53:57,842 INFO [Listener at localhost/35403] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/java.io.tmpdir/Jetty_localhost_39395_datanode____w46d3k/webapp 2023-02-11 00:53:57,945 INFO [Listener at localhost/35403] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39395 2023-02-11 00:53:57,954 WARN [Listener at localhost/35701] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 00:53:58,166 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x989981543986aca5: Processing first storage report for DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a from datanode 0b950d92-fb4b-4a88-add0-dd4dcb24d701 2023-02-11 00:53:58,168 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x989981543986aca5: from storage DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a node DatanodeRegistration(127.0.0.1:45981, datanodeUuid=0b950d92-fb4b-4a88-add0-dd4dcb24d701, infoPort=35019, infoSecurePort=0, ipcPort=35403, storageInfo=lv=-57;cid=testClusterID;nsid=1803545546;c=1676076836114), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-11 00:53:58,168 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x989981543986aca5: Processing first storage report for DS-a4c63464-e001-4a72-a4fe-e634251bc61b from datanode 0b950d92-fb4b-4a88-add0-dd4dcb24d701 2023-02-11 00:53:58,168 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x989981543986aca5: from storage DS-a4c63464-e001-4a72-a4fe-e634251bc61b node DatanodeRegistration(127.0.0.1:45981, datanodeUuid=0b950d92-fb4b-4a88-add0-dd4dcb24d701, infoPort=35019, infoSecurePort=0, ipcPort=35403, storageInfo=lv=-57;cid=testClusterID;nsid=1803545546;c=1676076836114), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 00:53:58,168 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x46f9345b38478928: Processing first storage report for DS-18e2de3f-76b7-4972-bc9c-e774f1f05107 from datanode 37dee797-9923-4df8-b47e-bcee8ee34413 2023-02-11 00:53:58,169 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x46f9345b38478928: from storage DS-18e2de3f-76b7-4972-bc9c-e774f1f05107 node DatanodeRegistration(127.0.0.1:43771, datanodeUuid=37dee797-9923-4df8-b47e-bcee8ee34413, infoPort=45113, infoSecurePort=0, ipcPort=44865, storageInfo=lv=-57;cid=testClusterID;nsid=1803545546;c=1676076836114), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 00:53:58,169 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x46f9345b38478928: Processing first storage report for DS-c7e74a4b-911e-46a6-9ea8-e497b260fb59 from datanode 37dee797-9923-4df8-b47e-bcee8ee34413 2023-02-11 00:53:58,169 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x46f9345b38478928: from storage DS-c7e74a4b-911e-46a6-9ea8-e497b260fb59 node DatanodeRegistration(127.0.0.1:43771, datanodeUuid=37dee797-9923-4df8-b47e-bcee8ee34413, infoPort=45113, infoSecurePort=0, ipcPort=44865, storageInfo=lv=-57;cid=testClusterID;nsid=1803545546;c=1676076836114), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 00:53:58,182 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4af05dabc1160a99: Processing first storage report for DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844 from datanode e08011bb-ec6b-4682-9527-eafae3012242 2023-02-11 00:53:58,182 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4af05dabc1160a99: from storage DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844 node DatanodeRegistration(127.0.0.1:39005, datanodeUuid=e08011bb-ec6b-4682-9527-eafae3012242, infoPort=41871, infoSecurePort=0, ipcPort=35701, storageInfo=lv=-57;cid=testClusterID;nsid=1803545546;c=1676076836114), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 00:53:58,182 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x4af05dabc1160a99: Processing first storage report for DS-974cea7d-d579-4429-be48-60dbb7e1b5f7 from datanode e08011bb-ec6b-4682-9527-eafae3012242 2023-02-11 00:53:58,182 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x4af05dabc1160a99: from storage DS-974cea7d-d579-4429-be48-60dbb7e1b5f7 node DatanodeRegistration(127.0.0.1:39005, datanodeUuid=e08011bb-ec6b-4682-9527-eafae3012242, infoPort=41871, infoSecurePort=0, ipcPort=35701, storageInfo=lv=-57;cid=testClusterID;nsid=1803545546;c=1676076836114), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 00:53:58,352 DEBUG [Listener at localhost/35701] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6 2023-02-11 00:53:58,420 INFO [Listener at localhost/35701] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/zookeeper_0, clientPort=56062, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-11 00:53:58,438 INFO [Listener at localhost/35701] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=56062 2023-02-11 00:53:58,446 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:53:58,447 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:53:59,101 INFO [Listener at localhost/35701] util.FSUtils(479): Created version file at hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89 with version=8 2023-02-11 00:53:59,102 INFO [Listener at localhost/35701] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/hbase-staging 2023-02-11 00:53:59,393 INFO [Listener at localhost/35701] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-11 00:53:59,848 INFO [Listener at localhost/35701] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 00:53:59,884 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 00:53:59,884 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 00:53:59,885 INFO [Listener at localhost/35701] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 00:53:59,885 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 00:53:59,885 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 00:54:00,030 INFO [Listener at localhost/35701] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 00:54:00,102 DEBUG [Listener at localhost/35701] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-11 00:54:00,197 INFO [Listener at localhost/35701] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46819 2023-02-11 00:54:00,207 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,209 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,230 INFO [Listener at localhost/35701] zookeeper.RecoverableZooKeeper(93): Process identifier=master:46819 connecting to ZooKeeper ensemble=127.0.0.1:56062 2023-02-11 00:54:00,271 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:468190x0, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 00:54:00,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:46819-0x10040ded7c50000 connected 2023-02-11 00:54:00,297 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 00:54:00,298 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 00:54:00,301 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 00:54:00,309 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46819 2023-02-11 00:54:00,310 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46819 2023-02-11 00:54:00,310 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46819 2023-02-11 00:54:00,311 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46819 2023-02-11 00:54:00,311 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46819 2023-02-11 00:54:00,317 INFO [Listener at localhost/35701] master.HMaster(439): hbase.rootdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89, hbase.cluster.distributed=false 2023-02-11 00:54:00,387 INFO [Listener at localhost/35701] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 00:54:00,387 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,387 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,387 INFO [Listener at localhost/35701] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 00:54:00,388 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,388 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 00:54:00,393 INFO [Listener at localhost/35701] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 00:54:00,395 INFO [Listener at localhost/35701] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33641 2023-02-11 00:54:00,398 INFO [Listener at localhost/35701] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 00:54:00,404 DEBUG [Listener at localhost/35701] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 00:54:00,405 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,407 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,408 INFO [Listener at localhost/35701] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33641 connecting to ZooKeeper ensemble=127.0.0.1:56062 2023-02-11 00:54:00,412 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:336410x0, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 00:54:00,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33641-0x10040ded7c50001 connected 2023-02-11 00:54:00,414 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 00:54:00,415 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 00:54:00,416 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 00:54:00,420 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33641 2023-02-11 00:54:00,420 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33641 2023-02-11 00:54:00,421 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33641 2023-02-11 00:54:00,421 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33641 2023-02-11 00:54:00,421 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33641 2023-02-11 00:54:00,434 INFO [Listener at localhost/35701] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 00:54:00,434 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,435 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,435 INFO [Listener at localhost/35701] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 00:54:00,435 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,435 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 00:54:00,436 INFO [Listener at localhost/35701] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 00:54:00,437 INFO [Listener at localhost/35701] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38317 2023-02-11 00:54:00,438 INFO [Listener at localhost/35701] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 00:54:00,439 DEBUG [Listener at localhost/35701] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 00:54:00,439 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,442 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,445 INFO [Listener at localhost/35701] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38317 connecting to ZooKeeper ensemble=127.0.0.1:56062 2023-02-11 00:54:00,448 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:383170x0, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 00:54:00,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:38317-0x10040ded7c50002 connected 2023-02-11 00:54:00,450 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 00:54:00,450 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 00:54:00,451 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 00:54:00,451 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38317 2023-02-11 00:54:00,452 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38317 2023-02-11 00:54:00,453 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38317 2023-02-11 00:54:00,455 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38317 2023-02-11 00:54:00,455 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38317 2023-02-11 00:54:00,468 INFO [Listener at localhost/35701] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 00:54:00,468 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,468 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,468 INFO [Listener at localhost/35701] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 00:54:00,469 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 00:54:00,469 INFO [Listener at localhost/35701] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 00:54:00,469 INFO [Listener at localhost/35701] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 00:54:00,470 INFO [Listener at localhost/35701] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41445 2023-02-11 00:54:00,470 INFO [Listener at localhost/35701] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 00:54:00,471 DEBUG [Listener at localhost/35701] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 00:54:00,472 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,474 INFO [Listener at localhost/35701] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,475 INFO [Listener at localhost/35701] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41445 connecting to ZooKeeper ensemble=127.0.0.1:56062 2023-02-11 00:54:00,478 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:414450x0, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 00:54:00,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41445-0x10040ded7c50003 connected 2023-02-11 00:54:00,480 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 00:54:00,480 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 00:54:00,481 DEBUG [Listener at localhost/35701] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 00:54:00,482 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41445 2023-02-11 00:54:00,482 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41445 2023-02-11 00:54:00,482 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41445 2023-02-11 00:54:00,483 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41445 2023-02-11 00:54:00,483 DEBUG [Listener at localhost/35701] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41445 2023-02-11 00:54:00,484 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:54:00,494 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 00:54:00,495 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:54:00,513 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 00:54:00,513 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 00:54:00,513 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 00:54:00,513 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 00:54:00,513 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:00,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 00:54:00,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,46819,1676076839233 from backup master directory 2023-02-11 00:54:00,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 00:54:00,521 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:54:00,521 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 00:54:00,521 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 00:54:00,522 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:54:00,524 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-11 00:54:00,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-11 00:54:00,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/hbase.id with ID: a0c30e39-1a4f-4a7a-aa54-9de59ca6b657 2023-02-11 00:54:00,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 00:54:00,672 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:00,732 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x29489d83 to 127.0.0.1:56062 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 00:54:00,780 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47b4d3d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 00:54:00,810 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 00:54:00,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-11 00:54:00,831 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-11 00:54:00,831 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-11 00:54:00,833 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 00:54:00,837 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 00:54:00,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 00:54:00,879 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store-tmp 2023-02-11 00:54:00,947 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:00,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 00:54:00,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 00:54:00,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 00:54:00,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 00:54:00,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 00:54:00,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 00:54:00,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676076840947Disabling compacts and flushes for region at 1676076840948 (+1 ms)Disabling writes for close at 1676076840949 (+1 ms)Writing region close event to WAL at 1676076840949Closed at 1676076840949 2023-02-11 00:54:00,951 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/WALs/jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:54:00,976 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46819%2C1676076839233, suffix=, logDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/WALs/jenkins-hbase4.apache.org,46819,1676076839233, archiveDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/oldWALs, maxLogs=10 2023-02-11 00:54:01,045 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK] 2023-02-11 00:54:01,045 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK] 2023-02-11 00:54:01,045 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK] 2023-02-11 00:54:01,054 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-11 00:54:01,151 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/WALs/jenkins-hbase4.apache.org,46819,1676076839233/jenkins-hbase4.apache.org%2C46819%2C1676076839233.1676076840991 2023-02-11 00:54:01,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK], DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK], DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK]] 2023-02-11 00:54:01,152 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:01,152 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:01,156 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 00:54:01,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 00:54:01,231 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-11 00:54:01,238 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-11 00:54:01,265 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-11 00:54:01,277 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:01,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 00:54:01,284 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 00:54:01,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 00:54:01,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:01,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67203399, jitterRate=0.0014086812734603882}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 00:54:01,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676076841187Initializing all the Stores at 1676076841190 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076841190Cleaning up temporary data from old regions at 1676076841290 (+100 ms)Cleaning up detritus from prior splits at 1676076841290Region opened successfully at 1676076841303 (+13 ms) 2023-02-11 00:54:01,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-11 00:54:01,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:01,323 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:01,325 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:01,326 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:01,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-11 00:54:01,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-11 00:54:01,381 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-11 00:54:01,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-11 00:54:01,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-11 00:54:01,412 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-11 00:54:01,413 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-11 00:54:01,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-11 00:54:01,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-11 00:54:01,424 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:01,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-11 00:54:01,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-11 00:54:01,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-11 00:54:01,443 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 00:54:01,443 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 00:54:01,443 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 00:54:01,443 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 00:54:01,443 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:01,443 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,46819,1676076839233, sessionid=0x10040ded7c50000, setting cluster-up flag (Was=false) 2023-02-11 00:54:01,459 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:01,463 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-11 00:54:01,465 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:54:01,470 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:01,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-11 00:54:01,478 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:54:01,481 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.hbase-snapshot/.tmp 2023-02-11 00:54:01,487 INFO [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(952): ClusterId : a0c30e39-1a4f-4a7a-aa54-9de59ca6b657 2023-02-11 00:54:01,487 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(952): ClusterId : a0c30e39-1a4f-4a7a-aa54-9de59ca6b657 2023-02-11 00:54:01,487 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(952): ClusterId : a0c30e39-1a4f-4a7a-aa54-9de59ca6b657 2023-02-11 00:54:01,492 DEBUG [RS:2;jenkins-hbase4:41445] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 00:54:01,492 DEBUG [RS:0;jenkins-hbase4:33641] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 00:54:01,492 DEBUG [RS:1;jenkins-hbase4:38317] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 00:54:01,499 DEBUG [RS:1;jenkins-hbase4:38317] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 00:54:01,499 DEBUG [RS:2;jenkins-hbase4:41445] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 00:54:01,499 DEBUG [RS:0;jenkins-hbase4:33641] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 00:54:01,499 DEBUG [RS:2;jenkins-hbase4:41445] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 00:54:01,499 DEBUG [RS:1;jenkins-hbase4:38317] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 00:54:01,499 DEBUG [RS:0;jenkins-hbase4:33641] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 00:54:01,504 DEBUG [RS:1;jenkins-hbase4:38317] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 00:54:01,504 DEBUG [RS:0;jenkins-hbase4:33641] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 00:54:01,506 DEBUG [RS:1;jenkins-hbase4:38317] zookeeper.ReadOnlyZKClient(139): Connect 0x0e6ed5cc to 127.0.0.1:56062 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 00:54:01,506 DEBUG [RS:0;jenkins-hbase4:33641] zookeeper.ReadOnlyZKClient(139): Connect 0x5ce197e5 to 127.0.0.1:56062 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 00:54:01,506 DEBUG [RS:2;jenkins-hbase4:41445] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 00:54:01,508 DEBUG [RS:2;jenkins-hbase4:41445] zookeeper.ReadOnlyZKClient(139): Connect 0x6db76ec6 to 127.0.0.1:56062 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 00:54:01,514 DEBUG [RS:1;jenkins-hbase4:38317] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@a93a71, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 00:54:01,515 DEBUG [RS:0;jenkins-hbase4:33641] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@52267209, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 00:54:01,515 DEBUG [RS:1;jenkins-hbase4:38317] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@496bebc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 00:54:01,515 DEBUG [RS:0;jenkins-hbase4:33641] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5619fc95, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 00:54:01,516 DEBUG [RS:2;jenkins-hbase4:41445] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@63709d00, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 00:54:01,517 DEBUG [RS:2;jenkins-hbase4:41445] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@674f05b6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 00:54:01,542 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:38317 2023-02-11 00:54:01,542 DEBUG [RS:2;jenkins-hbase4:41445] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:41445 2023-02-11 00:54:01,543 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33641 2023-02-11 00:54:01,547 INFO [RS:1;jenkins-hbase4:38317] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 00:54:01,547 INFO [RS:2;jenkins-hbase4:41445] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 00:54:01,547 INFO [RS:1;jenkins-hbase4:38317] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 00:54:01,547 INFO [RS:0;jenkins-hbase4:33641] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 00:54:01,547 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 00:54:01,547 INFO [RS:2;jenkins-hbase4:41445] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 00:54:01,548 INFO [RS:0;jenkins-hbase4:33641] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 00:54:01,548 DEBUG [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 00:54:01,548 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 00:54:01,550 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46819,1676076839233 with isa=jenkins-hbase4.apache.org/172.31.14.131:38317, startcode=1676076840434 2023-02-11 00:54:01,550 INFO [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46819,1676076839233 with isa=jenkins-hbase4.apache.org/172.31.14.131:41445, startcode=1676076840467 2023-02-11 00:54:01,550 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46819,1676076839233 with isa=jenkins-hbase4.apache.org/172.31.14.131:33641, startcode=1676076840386 2023-02-11 00:54:01,569 DEBUG [RS:1;jenkins-hbase4:38317] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 00:54:01,569 DEBUG [RS:2;jenkins-hbase4:41445] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 00:54:01,569 DEBUG [RS:0;jenkins-hbase4:33641] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 00:54:01,617 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:01,626 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50377, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 00:54:01,627 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44693, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 00:54:01,626 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50061, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 00:54:01,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 00:54:01,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 00:54:01,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 00:54:01,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 00:54:01,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-11 00:54:01,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 00:54:01,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076871632 2023-02-11 00:54:01,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-11 00:54:01,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 00:54:01,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46819] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 00:54:01,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46819] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 00:54:01,642 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:01,643 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-11 00:54:01,654 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 00:54:01,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-11 00:54:01,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-11 00:54:01,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-11 00:54:01,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-11 00:54:01,669 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-11 00:54:01,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-11 00:54:01,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-11 00:54:01,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-11 00:54:01,676 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 00:54:01,676 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 00:54:01,676 WARN [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 00:54:01,676 DEBUG [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 00:54:01,677 WARN [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 00:54:01,677 WARN [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 00:54:01,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-11 00:54:01,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-11 00:54:01,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676076841679,5,FailOnTimeoutGroup] 2023-02-11 00:54:01,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676076841680,5,FailOnTimeoutGroup] 2023-02-11 00:54:01,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,681 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-11 00:54:01,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,713 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:01,714 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:01,714 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89 2023-02-11 00:54:01,740 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:01,742 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 00:54:01,744 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/info 2023-02-11 00:54:01,745 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 00:54:01,746 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:01,746 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 00:54:01,748 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/rep_barrier 2023-02-11 00:54:01,749 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 00:54:01,749 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:01,750 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 00:54:01,751 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/table 2023-02-11 00:54:01,752 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 00:54:01,753 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:01,754 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740 2023-02-11 00:54:01,755 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740 2023-02-11 00:54:01,758 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 00:54:01,760 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 00:54:01,764 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:01,765 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65402910, jitterRate=-0.025420695543289185}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 00:54:01,765 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676076841740Initializing all the Stores at 1676076841742 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676076841742Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076841742Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676076841742Cleaning up temporary data from old regions at 1676076841756 (+14 ms)Cleaning up detritus from prior splits at 1676076841757 (+1 ms)Region opened successfully at 1676076841765 (+8 ms) 2023-02-11 00:54:01,766 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 00:54:01,766 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 00:54:01,766 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 00:54:01,766 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 00:54:01,766 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 00:54:01,767 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 00:54:01,767 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676076841766Disabling compacts and flushes for region at 1676076841766Disabling writes for close at 1676076841766Writing region close event to WAL at 1676076841767 (+1 ms)Closed at 1676076841767 2023-02-11 00:54:01,774 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:01,774 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-11 00:54:01,778 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46819,1676076839233 with isa=jenkins-hbase4.apache.org/172.31.14.131:38317, startcode=1676076840434 2023-02-11 00:54:01,779 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46819,1676076839233 with isa=jenkins-hbase4.apache.org/172.31.14.131:33641, startcode=1676076840386 2023-02-11 00:54:01,778 INFO [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,46819,1676076839233 with isa=jenkins-hbase4.apache.org/172.31.14.131:41445, startcode=1676076840467 2023-02-11 00:54:01,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46819] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:01,786 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:01,787 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46819] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:01,787 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-11 00:54:01,793 DEBUG [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89 2023-02-11 00:54:01,794 DEBUG [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:46775 2023-02-11 00:54:01,794 DEBUG [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 00:54:01,796 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89 2023-02-11 00:54:01,796 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89 2023-02-11 00:54:01,796 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:46775 2023-02-11 00:54:01,797 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:46775 2023-02-11 00:54:01,797 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 00:54:01,797 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 00:54:01,802 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 00:54:01,803 DEBUG [RS:1;jenkins-hbase4:38317] zookeeper.ZKUtil(162): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:01,803 DEBUG [RS:0;jenkins-hbase4:33641] zookeeper.ZKUtil(162): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:01,804 WARN [RS:1;jenkins-hbase4:38317] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 00:54:01,804 WARN [RS:0;jenkins-hbase4:33641] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 00:54:01,805 INFO [RS:1;jenkins-hbase4:38317] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 00:54:01,805 INFO [RS:0;jenkins-hbase4:33641] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 00:54:01,805 DEBUG [RS:2;jenkins-hbase4:41445] zookeeper.ZKUtil(162): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:01,805 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1947): logDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:01,805 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1947): logDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:01,805 WARN [RS:2;jenkins-hbase4:41445] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 00:54:01,806 INFO [RS:2;jenkins-hbase4:41445] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 00:54:01,806 DEBUG [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1947): logDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:01,806 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38317,1676076840434] 2023-02-11 00:54:01,806 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41445,1676076840467] 2023-02-11 00:54:01,806 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33641,1676076840386] 2023-02-11 00:54:01,812 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-11 00:54:01,817 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 00:54:01,820 DEBUG [RS:2;jenkins-hbase4:41445] zookeeper.ZKUtil(162): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:01,820 DEBUG [RS:0;jenkins-hbase4:33641] zookeeper.ZKUtil(162): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:01,820 DEBUG [RS:1;jenkins-hbase4:38317] zookeeper.ZKUtil(162): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:01,821 DEBUG [RS:2;jenkins-hbase4:41445] zookeeper.ZKUtil(162): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:01,821 DEBUG [RS:0;jenkins-hbase4:33641] zookeeper.ZKUtil(162): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:01,821 DEBUG [RS:1;jenkins-hbase4:38317] zookeeper.ZKUtil(162): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:01,822 DEBUG [RS:2;jenkins-hbase4:41445] zookeeper.ZKUtil(162): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:01,822 DEBUG [RS:0;jenkins-hbase4:33641] zookeeper.ZKUtil(162): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:01,822 DEBUG [RS:1;jenkins-hbase4:38317] zookeeper.ZKUtil(162): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:01,833 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 00:54:01,833 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 00:54:01,833 DEBUG [RS:2;jenkins-hbase4:41445] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 00:54:01,842 INFO [RS:2;jenkins-hbase4:41445] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 00:54:01,842 INFO [RS:1;jenkins-hbase4:38317] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 00:54:01,842 INFO [RS:0;jenkins-hbase4:33641] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 00:54:01,863 INFO [RS:0;jenkins-hbase4:33641] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 00:54:01,863 INFO [RS:2;jenkins-hbase4:41445] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 00:54:01,863 INFO [RS:1;jenkins-hbase4:38317] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 00:54:01,868 INFO [RS:1;jenkins-hbase4:38317] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 00:54:01,868 INFO [RS:1;jenkins-hbase4:38317] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,868 INFO [RS:2;jenkins-hbase4:41445] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 00:54:01,869 INFO [RS:0;jenkins-hbase4:33641] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 00:54:01,869 INFO [RS:2;jenkins-hbase4:41445] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,869 INFO [RS:0;jenkins-hbase4:33641] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,871 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 00:54:01,871 INFO [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 00:54:01,871 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 00:54:01,879 INFO [RS:1;jenkins-hbase4:38317] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,879 INFO [RS:2;jenkins-hbase4:41445] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,879 INFO [RS:0;jenkins-hbase4:33641] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,879 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,879 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,881 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 00:54:01,881 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 00:54:01,881 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,881 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,881 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,881 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,881 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,881 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,881 DEBUG [RS:1;jenkins-hbase4:38317] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,881 DEBUG [RS:0;jenkins-hbase4:33641] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,880 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,882 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,882 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,882 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,883 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 00:54:01,883 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,883 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,883 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,883 DEBUG [RS:2;jenkins-hbase4:41445] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 00:54:01,886 INFO [RS:1;jenkins-hbase4:38317] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,886 INFO [RS:1;jenkins-hbase4:38317] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,886 INFO [RS:1;jenkins-hbase4:38317] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,887 INFO [RS:0;jenkins-hbase4:33641] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,887 INFO [RS:0;jenkins-hbase4:33641] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,887 INFO [RS:0;jenkins-hbase4:33641] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,888 INFO [RS:2;jenkins-hbase4:41445] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,888 INFO [RS:2;jenkins-hbase4:41445] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,888 INFO [RS:2;jenkins-hbase4:41445] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,907 INFO [RS:2;jenkins-hbase4:41445] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 00:54:01,907 INFO [RS:0;jenkins-hbase4:33641] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 00:54:01,907 INFO [RS:1;jenkins-hbase4:38317] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 00:54:01,910 INFO [RS:0;jenkins-hbase4:33641] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33641,1676076840386-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,910 INFO [RS:2;jenkins-hbase4:41445] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41445,1676076840467-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,910 INFO [RS:1;jenkins-hbase4:38317] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38317,1676076840434-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:01,934 INFO [RS:2;jenkins-hbase4:41445] regionserver.Replication(203): jenkins-hbase4.apache.org,41445,1676076840467 started 2023-02-11 00:54:01,934 INFO [RS:0;jenkins-hbase4:33641] regionserver.Replication(203): jenkins-hbase4.apache.org,33641,1676076840386 started 2023-02-11 00:54:01,934 INFO [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,41445,1676076840467, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41445, sessionid=0x10040ded7c50003 2023-02-11 00:54:01,934 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33641,1676076840386, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33641, sessionid=0x10040ded7c50001 2023-02-11 00:54:01,936 INFO [RS:1;jenkins-hbase4:38317] regionserver.Replication(203): jenkins-hbase4.apache.org,38317,1676076840434 started 2023-02-11 00:54:01,936 DEBUG [RS:2;jenkins-hbase4:41445] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 00:54:01,936 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,38317,1676076840434, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38317, sessionid=0x10040ded7c50002 2023-02-11 00:54:01,936 DEBUG [RS:2;jenkins-hbase4:41445] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:01,936 DEBUG [RS:1;jenkins-hbase4:38317] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 00:54:01,936 DEBUG [RS:2;jenkins-hbase4:41445] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41445,1676076840467' 2023-02-11 00:54:01,936 DEBUG [RS:0;jenkins-hbase4:33641] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 00:54:01,937 DEBUG [RS:2;jenkins-hbase4:41445] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:01,936 DEBUG [RS:1;jenkins-hbase4:38317] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:01,937 DEBUG [RS:0;jenkins-hbase4:33641] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:01,937 DEBUG [RS:1;jenkins-hbase4:38317] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38317,1676076840434' 2023-02-11 00:54:01,938 DEBUG [RS:1;jenkins-hbase4:38317] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:01,938 DEBUG [RS:0;jenkins-hbase4:33641] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33641,1676076840386' 2023-02-11 00:54:01,938 DEBUG [RS:0;jenkins-hbase4:33641] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:01,939 DEBUG [RS:2;jenkins-hbase4:41445] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:01,939 DEBUG [RS:1;jenkins-hbase4:38317] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:01,939 DEBUG [RS:0;jenkins-hbase4:33641] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:01,940 DEBUG [RS:2;jenkins-hbase4:41445] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 00:54:01,940 DEBUG [RS:2;jenkins-hbase4:41445] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 00:54:01,940 DEBUG [RS:1;jenkins-hbase4:38317] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 00:54:01,940 DEBUG [RS:0;jenkins-hbase4:33641] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 00:54:01,940 DEBUG [RS:1;jenkins-hbase4:38317] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 00:54:01,940 DEBUG [RS:2;jenkins-hbase4:41445] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:01,940 DEBUG [RS:1;jenkins-hbase4:38317] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:01,941 DEBUG [RS:1;jenkins-hbase4:38317] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38317,1676076840434' 2023-02-11 00:54:01,941 DEBUG [RS:1;jenkins-hbase4:38317] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 00:54:01,940 DEBUG [RS:0;jenkins-hbase4:33641] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 00:54:01,941 DEBUG [RS:2;jenkins-hbase4:41445] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41445,1676076840467' 2023-02-11 00:54:01,941 DEBUG [RS:2;jenkins-hbase4:41445] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 00:54:01,941 DEBUG [RS:0;jenkins-hbase4:33641] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:01,941 DEBUG [RS:0;jenkins-hbase4:33641] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33641,1676076840386' 2023-02-11 00:54:01,941 DEBUG [RS:0;jenkins-hbase4:33641] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 00:54:01,941 DEBUG [RS:1;jenkins-hbase4:38317] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 00:54:01,942 DEBUG [RS:0;jenkins-hbase4:33641] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 00:54:01,942 DEBUG [RS:2;jenkins-hbase4:41445] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 00:54:01,942 DEBUG [RS:1;jenkins-hbase4:38317] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 00:54:01,942 INFO [RS:1;jenkins-hbase4:38317] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 00:54:01,942 INFO [RS:1;jenkins-hbase4:38317] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 00:54:01,942 DEBUG [RS:0;jenkins-hbase4:33641] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 00:54:01,943 DEBUG [RS:2;jenkins-hbase4:41445] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 00:54:01,943 INFO [RS:0;jenkins-hbase4:33641] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 00:54:01,943 INFO [RS:2;jenkins-hbase4:41445] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 00:54:01,943 INFO [RS:0;jenkins-hbase4:33641] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 00:54:01,944 INFO [RS:2;jenkins-hbase4:41445] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 00:54:01,970 DEBUG [jenkins-hbase4:46819] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-11 00:54:01,973 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:01,979 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:01,979 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:01,979 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:01,980 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:54:01,982 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33641,1676076840386, state=OPENING 2023-02-11 00:54:01,990 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-11 00:54:01,991 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:01,992 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 00:54:01,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:02,053 INFO [RS:2;jenkins-hbase4:41445] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41445%2C1676076840467, suffix=, logDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,41445,1676076840467, archiveDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/oldWALs, maxLogs=32 2023-02-11 00:54:02,053 INFO [RS:0;jenkins-hbase4:33641] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33641%2C1676076840386, suffix=, logDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,33641,1676076840386, archiveDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/oldWALs, maxLogs=32 2023-02-11 00:54:02,053 INFO [RS:1;jenkins-hbase4:38317] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38317%2C1676076840434, suffix=, logDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,38317,1676076840434, archiveDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/oldWALs, maxLogs=32 2023-02-11 00:54:02,078 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK] 2023-02-11 00:54:02,079 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK] 2023-02-11 00:54:02,079 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK] 2023-02-11 00:54:02,091 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK] 2023-02-11 00:54:02,094 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK] 2023-02-11 00:54:02,094 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK] 2023-02-11 00:54:02,096 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK] 2023-02-11 00:54:02,096 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK] 2023-02-11 00:54:02,096 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK] 2023-02-11 00:54:02,098 INFO [RS:0;jenkins-hbase4:33641] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,33641,1676076840386/jenkins-hbase4.apache.org%2C33641%2C1676076840386.1676076842060 2023-02-11 00:54:02,099 DEBUG [RS:0;jenkins-hbase4:33641] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK], DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK], DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK]] 2023-02-11 00:54:02,103 INFO [RS:2;jenkins-hbase4:41445] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,41445,1676076840467/jenkins-hbase4.apache.org%2C41445%2C1676076840467.1676076842060 2023-02-11 00:54:02,103 INFO [RS:1;jenkins-hbase4:38317] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,38317,1676076840434/jenkins-hbase4.apache.org%2C38317%2C1676076840434.1676076842060 2023-02-11 00:54:02,104 DEBUG [RS:2;jenkins-hbase4:41445] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK], DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK], DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK]] 2023-02-11 00:54:02,104 DEBUG [RS:1;jenkins-hbase4:38317] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK], DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK], DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK]] 2023-02-11 00:54:02,183 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:02,185 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 00:54:02,189 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56314, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 00:54:02,203 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-11 00:54:02,203 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 00:54:02,207 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33641%2C1676076840386.meta, suffix=.meta, logDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,33641,1676076840386, archiveDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/oldWALs, maxLogs=32 2023-02-11 00:54:02,227 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK] 2023-02-11 00:54:02,229 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK] 2023-02-11 00:54:02,230 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK] 2023-02-11 00:54:02,236 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/WALs/jenkins-hbase4.apache.org,33641,1676076840386/jenkins-hbase4.apache.org%2C33641%2C1676076840386.meta.1676076842208.meta 2023-02-11 00:54:02,238 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:39005,DS-0f1c0c2b-dc1c-41e6-a35a-5271af81c844,DISK], DatanodeInfoWithStorage[127.0.0.1:43771,DS-18e2de3f-76b7-4972-bc9c-e774f1f05107,DISK], DatanodeInfoWithStorage[127.0.0.1:45981,DS-8ef078f0-1405-4a2b-a8ed-d6c00036b36a,DISK]] 2023-02-11 00:54:02,238 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:02,243 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:02,244 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-11 00:54:02,260 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-11 00:54:02,263 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-11 00:54:02,268 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-11 00:54:02,268 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:02,268 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-11 00:54:02,268 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-11 00:54:02,271 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 00:54:02,273 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/info 2023-02-11 00:54:02,273 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/info 2023-02-11 00:54:02,273 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 00:54:02,274 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:02,274 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 00:54:02,276 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/rep_barrier 2023-02-11 00:54:02,276 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/rep_barrier 2023-02-11 00:54:02,276 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 00:54:02,277 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:02,277 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 00:54:02,279 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/table 2023-02-11 00:54:02,279 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/table 2023-02-11 00:54:02,279 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 00:54:02,280 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:02,282 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740 2023-02-11 00:54:02,285 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740 2023-02-11 00:54:02,288 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 00:54:02,291 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 00:54:02,292 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68289362, jitterRate=0.01759079098701477}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 00:54:02,292 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676076842268Writing region info on filesystem at 1676076842269 (+1 ms)Initializing all the Stores at 1676076842270 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676076842270Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076842271 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676076842271Cleaning up temporary data from old regions at 1676076842287 (+16 ms)Cleaning up detritus from prior splits at 1676076842287Running coprocessor post-open hooks at 1676076842292 (+5 ms)Region opened successfully at 1676076842292 2023-02-11 00:54:02,302 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676076842175 2023-02-11 00:54:02,321 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-11 00:54:02,322 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-11 00:54:02,323 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33641,1676076840386, state=OPEN 2023-02-11 00:54:02,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-11 00:54:02,329 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 00:54:02,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-11 00:54:02,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33641,1676076840386 in 331 msec 2023-02-11 00:54:02,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-11 00:54:02,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 553 msec 2023-02-11 00:54:02,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 813 msec 2023-02-11 00:54:02,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676076840509, completionTime=-1 2023-02-11 00:54:02,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-11 00:54:02,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:02,423 DEBUG [hconnection-0x79353ebb-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 00:54:02,427 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56320, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 00:54:02,453 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:02,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076902454 2023-02-11 00:54:02,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076962454 2023-02-11 00:54:02,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 99 msec 2023-02-11 00:54:02,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46819,1676076839233-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:02,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46819,1676076839233-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:02,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46819,1676076839233-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:02,480 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:46819, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:02,481 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-11 00:54:02,487 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-11 00:54:02,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-11 00:54:02,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 00:54:02,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:02,509 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:54:02,512 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:54:02,533 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:02,535 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f empty. 2023-02-11 00:54:02,536 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:02,536 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-11 00:54:02,575 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:02,577 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => b9396c6e19f6c2ad076604e11fcfef4f, NAME => 'hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:54:02,597 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:02,597 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing b9396c6e19f6c2ad076604e11fcfef4f, disabling compactions & flushes 2023-02-11 00:54:02,597 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:54:02,597 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:54:02,597 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. after waiting 0 ms 2023-02-11 00:54:02,597 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:54:02,597 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:54:02,597 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for b9396c6e19f6c2ad076604e11fcfef4f: Waiting for close lock at 1676076842597Disabling compacts and flushes for region at 1676076842597Disabling writes for close at 1676076842597Writing region close event to WAL at 1676076842597Closed at 1676076842597 2023-02-11 00:54:02,602 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:54:02,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676076842605"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076842605"}]},"ts":"1676076842605"} 2023-02-11 00:54:02,645 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:54:02,646 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:54:02,652 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076842647"}]},"ts":"1676076842647"} 2023-02-11 00:54:02,656 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-11 00:54:02,661 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:02,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:02,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:02,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:02,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:54:02,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b9396c6e19f6c2ad076604e11fcfef4f, ASSIGN}] 2023-02-11 00:54:02,668 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b9396c6e19f6c2ad076604e11fcfef4f, ASSIGN 2023-02-11 00:54:02,670 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=b9396c6e19f6c2ad076604e11fcfef4f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:02,822 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:02,823 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:02,823 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676076842823"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076842823"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076842823"}]},"ts":"1676076842823"} 2023-02-11 00:54:02,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure b9396c6e19f6c2ad076604e11fcfef4f, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:02,982 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:02,982 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 00:54:02,985 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45938, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 00:54:02,991 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:54:02,991 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b9396c6e19f6c2ad076604e11fcfef4f, NAME => 'hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:02,992 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:02,993 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:02,993 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:02,993 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:02,993 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:02,995 INFO [StoreOpener-b9396c6e19f6c2ad076604e11fcfef4f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:02,997 DEBUG [StoreOpener-b9396c6e19f6c2ad076604e11fcfef4f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f/info 2023-02-11 00:54:02,997 DEBUG [StoreOpener-b9396c6e19f6c2ad076604e11fcfef4f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f/info 2023-02-11 00:54:02,997 INFO [StoreOpener-b9396c6e19f6c2ad076604e11fcfef4f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9396c6e19f6c2ad076604e11fcfef4f columnFamilyName info 2023-02-11 00:54:02,998 INFO [StoreOpener-b9396c6e19f6c2ad076604e11fcfef4f-1] regionserver.HStore(310): Store=b9396c6e19f6c2ad076604e11fcfef4f/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:03,000 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:03,001 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:03,005 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:54:03,009 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:03,010 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b9396c6e19f6c2ad076604e11fcfef4f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67697173, jitterRate=0.00876648724079132}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 00:54:03,010 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b9396c6e19f6c2ad076604e11fcfef4f: Running coprocessor pre-open hook at 1676076842993Writing region info on filesystem at 1676076842993Initializing all the Stores at 1676076842994 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676076842995 (+1 ms)Cleaning up temporary data from old regions at 1676076843002 (+7 ms)Cleaning up detritus from prior splits at 1676076843002Running coprocessor post-open hooks at 1676076843010 (+8 ms)Region opened successfully at 1676076843010 2023-02-11 00:54:03,012 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f., pid=6, masterSystemTime=1676076842982 2023-02-11 00:54:03,016 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:54:03,016 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:54:03,018 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:03,019 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676076843017"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076843017"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076843017"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076843017"}]},"ts":"1676076843017"} 2023-02-11 00:54:03,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-11 00:54:03,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure b9396c6e19f6c2ad076604e11fcfef4f, server=jenkins-hbase4.apache.org,38317,1676076840434 in 195 msec 2023-02-11 00:54:03,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-11 00:54:03,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=b9396c6e19f6c2ad076604e11fcfef4f, ASSIGN in 362 msec 2023-02-11 00:54:03,032 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:54:03,032 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076843032"}]},"ts":"1676076843032"} 2023-02-11 00:54:03,034 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-11 00:54:03,037 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:54:03,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 539 msec 2023-02-11 00:54:03,109 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-11 00:54:03,111 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-11 00:54:03,111 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:03,132 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 00:54:03,134 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45948, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 00:54:03,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:03,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 00:54:03,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 34 msec 2023-02-11 00:54:03,185 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:03,196 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 00:54:03,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-02-11 00:54:03,212 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-11 00:54:03,214 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-11 00:54:03,214 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.692sec 2023-02-11 00:54:03,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-11 00:54:03,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-11 00:54:03,219 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-11 00:54:03,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46819,1676076839233-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-11 00:54:03,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46819,1676076839233-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-11 00:54:03,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-11 00:54:03,294 DEBUG [Listener at localhost/35701] zookeeper.ReadOnlyZKClient(139): Connect 0x0c8d5aec to 127.0.0.1:56062 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 00:54:03,299 DEBUG [Listener at localhost/35701] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ff608fa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 00:54:03,312 DEBUG [hconnection-0x4d838b63-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 00:54:03,323 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56330, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 00:54:03,335 INFO [Listener at localhost/35701] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:54:03,366 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=363, OpenFileDescriptor=700, MaxFileDescriptor=60000, SystemLoadAverage=247, ProcessCount=168, AvailableMemoryMB=3582 2023-02-11 00:54:03,371 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:03,379 DEBUG [Listener at localhost/35701] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 00:54:03,382 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49384, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 00:54:03,396 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:54:03,396 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:54:03,397 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:54:03,403 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,405 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:54:03,407 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:54:03,410 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,411 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7 empty. 2023-02-11 00:54:03,413 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,413 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-11 00:54:03,434 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:03,436 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => b0843d8ffc44a9678fe6589a79fd74e7, NAME => 'testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:54:03,452 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:03,453 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing b0843d8ffc44a9678fe6589a79fd74e7, disabling compactions & flushes 2023-02-11 00:54:03,453 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:03,453 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:03,453 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. after waiting 0 ms 2023-02-11 00:54:03,453 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:03,453 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:03,453 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for b0843d8ffc44a9678fe6589a79fd74e7: Waiting for close lock at 1676076843453Disabling compacts and flushes for region at 1676076843453Disabling writes for close at 1676076843453Writing region close event to WAL at 1676076843453Closed at 1676076843453 2023-02-11 00:54:03,457 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:54:03,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676076843458"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076843458"}]},"ts":"1676076843458"} 2023-02-11 00:54:03,461 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:54:03,462 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:54:03,463 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076843462"}]},"ts":"1676076843462"} 2023-02-11 00:54:03,465 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-11 00:54:03,469 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:03,470 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:03,470 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:03,470 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:03,470 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:54:03,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b0843d8ffc44a9678fe6589a79fd74e7, ASSIGN}] 2023-02-11 00:54:03,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b0843d8ffc44a9678fe6589a79fd74e7, ASSIGN 2023-02-11 00:54:03,475 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b0843d8ffc44a9678fe6589a79fd74e7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:03,625 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:03,626 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b0843d8ffc44a9678fe6589a79fd74e7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:03,626 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676076843626"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076843626"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076843626"}]},"ts":"1676076843626"} 2023-02-11 00:54:03,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure b0843d8ffc44a9678fe6589a79fd74e7, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:03,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:03,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b0843d8ffc44a9678fe6589a79fd74e7, NAME => 'testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:03,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:03,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:03,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,792 INFO [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,794 DEBUG [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1 2023-02-11 00:54:03,794 DEBUG [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1 2023-02-11 00:54:03,794 INFO [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b0843d8ffc44a9678fe6589a79fd74e7 columnFamilyName cf1 2023-02-11 00:54:03,795 INFO [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] regionserver.HStore(310): Store=b0843d8ffc44a9678fe6589a79fd74e7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:03,795 INFO [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,797 DEBUG [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2 2023-02-11 00:54:03,797 DEBUG [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2 2023-02-11 00:54:03,798 INFO [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b0843d8ffc44a9678fe6589a79fd74e7 columnFamilyName cf2 2023-02-11 00:54:03,798 INFO [StoreOpener-b0843d8ffc44a9678fe6589a79fd74e7-1] regionserver.HStore(310): Store=b0843d8ffc44a9678fe6589a79fd74e7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:03,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:03,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:03,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:03,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b0843d8ffc44a9678fe6589a79fd74e7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62406899, jitterRate=-0.0700647383928299}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:03,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b0843d8ffc44a9678fe6589a79fd74e7: Running coprocessor pre-open hook at 1676076843790Writing region info on filesystem at 1676076843790Initializing all the Stores at 1676076843792 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076843792Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076843792Cleaning up temporary data from old regions at 1676076843802 (+10 ms)Cleaning up detritus from prior splits at 1676076843803 (+1 ms)Running coprocessor post-open hooks at 1676076843810 (+7 ms)Region opened successfully at 1676076843810 2023-02-11 00:54:03,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7., pid=11, masterSystemTime=1676076843782 2023-02-11 00:54:03,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:03,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:03,817 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b0843d8ffc44a9678fe6589a79fd74e7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:03,818 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676076843817"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076843817"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076843817"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076843817"}]},"ts":"1676076843817"} 2023-02-11 00:54:03,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-11 00:54:03,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure b0843d8ffc44a9678fe6589a79fd74e7, server=jenkins-hbase4.apache.org,38317,1676076840434 in 191 msec 2023-02-11 00:54:03,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-11 00:54:03,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b0843d8ffc44a9678fe6589a79fd74e7, ASSIGN in 353 msec 2023-02-11 00:54:03,828 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:54:03,828 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076843828"}]},"ts":"1676076843828"} 2023-02-11 00:54:03,830 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-11 00:54:03,834 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:54:03,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 433 msec 2023-02-11 00:54:03,908 DEBUG [Listener at localhost/35701] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 00:54:03,911 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45472, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 00:54:03,929 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:03,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,937 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:03,937 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:03,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 00:54:03,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 00:54:03,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,939 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:03,939 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:03,940 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:03,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:03,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:03,939 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:03,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:03,940 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:03,940 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:03,944 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:03,945 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:03,946 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:03,946 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:03,946 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:03,946 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. started... 2023-02-11 00:54:03,947 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b0843d8ffc44a9678fe6589a79fd74e7 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 00:54:03,948 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,948 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,948 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,948 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:03,948 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:03,948 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,024 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/907055a24ff04fb7a5aa8f20cbf36bbc 2023-02-11 00:54:04,087 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/c2d4c491a3e74530888035bb12adfb04 2023-02-11 00:54:04,097 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/907055a24ff04fb7a5aa8f20cbf36bbc as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc 2023-02-11 00:54:04,106 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:54:04,110 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/c2d4c491a3e74530888035bb12adfb04 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04 2023-02-11 00:54:04,119 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:54:04,121 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b0843d8ffc44a9678fe6589a79fd74e7 in 174ms, sequenceid=6, compaction requested=false 2023-02-11 00:54:04,123 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:04,125 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b0843d8ffc44a9678fe6589a79fd74e7: 2023-02-11 00:54:04,125 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,126 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,126 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,127 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,127 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,128 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 00:54:04,131 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:04,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@218b5976[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:04,131 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,136 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,136 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:04,140 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,140 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,140 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,140 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,141 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,141 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,141 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,141 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,141 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,141 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:04,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:04,146 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 00:54:04,146 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:04,146 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:04,146 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,146 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:04,148 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,148 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,148 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,148 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,148 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,148 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,149 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,149 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,151 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,152 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,154 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,155 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,156 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,167 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,167 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,168 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,169 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,169 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,169 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,169 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:04,169 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:04,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,172 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:04,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,172 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:04,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,272 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:04,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:04,303 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:04,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:04,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,306 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:04,306 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 00:54:04,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 00:54:04,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,309 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,309 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,309 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,309 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,309 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,309 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,312 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,312 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,312 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,312 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,320 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,320 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,320 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,320 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,320 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,320 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,320 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:04,321 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,321 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:04,321 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. started... 2023-02-11 00:54:04,322 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b0843d8ffc44a9678fe6589a79fd74e7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:04,323 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,324 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,324 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,324 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,325 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,325 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,341 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/e75819f609ce44fbbe4affb942bdc978 2023-02-11 00:54:04,370 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/67f05b137d6543089693339e96d9746b 2023-02-11 00:54:04,379 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/e75819f609ce44fbbe4affb942bdc978 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978 2023-02-11 00:54:04,387 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:54:04,388 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/67f05b137d6543089693339e96d9746b as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b 2023-02-11 00:54:04,396 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:54:04,397 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b0843d8ffc44a9678fe6589a79fd74e7 in 75ms, sequenceid=19, compaction requested=false 2023-02-11 00:54:04,397 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b0843d8ffc44a9678fe6589a79fd74e7: 2023-02-11 00:54:04,397 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:04,397 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:04,400 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:04,400 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,400 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,400 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,403 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,403 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,404 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,404 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 00:54:04,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@624f43e[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:04,407 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:04,407 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,408 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,408 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,408 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,409 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,409 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:04,411 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,411 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,411 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,411 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,411 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,412 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,412 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,412 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,412 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,412 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:04,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:04,417 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 00:54:04,417 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:04,417 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:04,417 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,417 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:04,420 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,420 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,420 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,420 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,420 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,420 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,421 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,421 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,422 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,423 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,429 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,429 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,430 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,438 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,438 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,438 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,439 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,438 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,439 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,438 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,439 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,439 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,439 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,439 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:04,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,439 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:04,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,439 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,439 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,439 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,440 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,440 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,440 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,440 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,441 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,441 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:04,441 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:04,541 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:04,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:04,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:04,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:04,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,568 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:04,568 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,569 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 00:54:04,569 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 00:54:04,569 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,569 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,571 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,571 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,571 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,571 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,571 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,571 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:04,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,572 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,573 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,573 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,573 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,573 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,573 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,573 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,573 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,573 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,574 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,574 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,573 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,574 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:04,574 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:04,574 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,574 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,575 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,574 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:04,575 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,575 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,575 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,575 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,575 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,575 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. started... 2023-02-11 00:54:04,575 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,575 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b0843d8ffc44a9678fe6589a79fd74e7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:04,578 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,579 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,579 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,579 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,579 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,579 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,597 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/ff24e1efe2e444d28c962ad988232caf 2023-02-11 00:54:04,619 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/c9f3035466c04d68a5923bdffcc19eb7 2023-02-11 00:54:04,627 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/ff24e1efe2e444d28c962ad988232caf as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf 2023-02-11 00:54:04,634 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:54:04,635 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/c9f3035466c04d68a5923bdffcc19eb7 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7 2023-02-11 00:54:04,642 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:54:04,643 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b0843d8ffc44a9678fe6589a79fd74e7 in 68ms, sequenceid=32, compaction requested=true 2023-02-11 00:54:04,643 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b0843d8ffc44a9678fe6589a79fd74e7: 2023-02-11 00:54:04,643 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:04,643 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:04,644 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:04,644 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,644 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,644 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,646 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,646 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,646 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,646 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 00:54:04,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24132b8d[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:04,649 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:04,649 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,650 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,650 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,650 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,651 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,651 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:04,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:04,651 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:04,655 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,655 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,655 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,656 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,656 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,656 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,656 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,656 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,656 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,656 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:04,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:04,661 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 00:54:04,662 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:04,662 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:04,662 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,662 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:04,664 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,664 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,664 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,664 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,664 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,665 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,665 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,668 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,669 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,669 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,672 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,672 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,672 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,672 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,673 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,672 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,673 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,673 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:04,673 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:04,673 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,673 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,673 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,674 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,674 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,674 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,674 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:04,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,674 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,674 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:04,675 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,675 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,675 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,675 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,775 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:04,776 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:04,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:04,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:04,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,801 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:04,801 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,802 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 00:54:04,802 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 00:54:04,803 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,803 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,805 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,805 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,805 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,805 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,805 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,806 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:04,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,807 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,807 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,808 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,808 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,809 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,809 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,809 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:04,809 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,809 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,809 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,809 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:04,809 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:04,810 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:04,810 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:04,811 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. started... 2023-02-11 00:54:04,811 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b0843d8ffc44a9678fe6589a79fd74e7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:04,812 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,813 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,833 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/abfb1b588828418cbd77c456c4781707 2023-02-11 00:54:04,858 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/8aad1841ef1040e68a6caea9bf21b53c 2023-02-11 00:54:04,867 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/abfb1b588828418cbd77c456c4781707 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707 2023-02-11 00:54:04,874 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:54:04,876 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/8aad1841ef1040e68a6caea9bf21b53c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c 2023-02-11 00:54:04,884 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:54:04,885 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b0843d8ffc44a9678fe6589a79fd74e7 in 74ms, sequenceid=45, compaction requested=true 2023-02-11 00:54:04,885 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b0843d8ffc44a9678fe6589a79fd74e7: 2023-02-11 00:54:04,885 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:04,885 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:04,886 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:04,886 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,886 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:04,886 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:04,893 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,893 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,893 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,893 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:04,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 00:54:04,896 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@472969c7[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:04,896 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:04,896 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:04,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:04,897 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:04,897 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:04,899 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,899 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,899 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,900 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,900 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,900 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,901 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:04,901 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,902 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,902 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:04,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:04,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:04,906 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 00:54:04,906 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:04,906 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:04,906 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,906 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:04,907 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,907 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:04,907 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,907 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,907 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,908 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,908 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,908 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,909 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,909 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:04,911 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,911 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,912 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:04,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,915 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:04,916 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,915 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:04,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:04,916 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:04,916 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:04,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,916 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:04,917 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:04,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:04,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:04,917 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,917 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:04,917 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:04,917 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:04,917 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,017 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:05,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:05,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:05,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:05,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,041 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:05,041 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:05,042 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 00:54:05,042 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 00:54:05,042 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,042 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,043 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,044 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,044 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,045 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,046 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:05,046 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:05,046 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:05,047 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:05,047 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:05,047 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,048 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:05,050 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:05,051 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:05,051 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:05,051 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:05,051 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. started... 2023-02-11 00:54:05,051 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b0843d8ffc44a9678fe6589a79fd74e7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:05,053 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,054 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,054 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,054 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:05,054 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,054 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:05,069 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/884619e6c3ea4ca4b866c5bf56209a53 2023-02-11 00:54:05,097 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/db42d0cbc71340bea870a75ad28dfcff 2023-02-11 00:54:05,105 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/884619e6c3ea4ca4b866c5bf56209a53 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/884619e6c3ea4ca4b866c5bf56209a53 2023-02-11 00:54:05,112 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/884619e6c3ea4ca4b866c5bf56209a53, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:54:05,113 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/db42d0cbc71340bea870a75ad28dfcff as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/db42d0cbc71340bea870a75ad28dfcff 2023-02-11 00:54:05,119 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/db42d0cbc71340bea870a75ad28dfcff, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:54:05,120 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b0843d8ffc44a9678fe6589a79fd74e7 in 69ms, sequenceid=58, compaction requested=true 2023-02-11 00:54:05,120 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b0843d8ffc44a9678fe6589a79fd74e7: 2023-02-11 00:54:05,120 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:05,120 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:05,121 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:05,121 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,121 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:05,121 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:05,123 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,123 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:05,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:05,123 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,123 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:05,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:05,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:05,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 00:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c12d8bd[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:05,126 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:05,126 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:05,127 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,127 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:05,127 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:05,130 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:05,130 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:05,130 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:05,131 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:05,131 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:05,131 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:05,132 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:05,132 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:05,132 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,132 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:05,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:05,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:05,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:05,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:05,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:05,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:05,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:05,138 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 00:54:05,139 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:05,139 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:05,139 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,139 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:05,140 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,140 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,140 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,140 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:05,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:05,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,141 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,140 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,141 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,141 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:05,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,142 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,142 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:05,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,144 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:05,145 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,145 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,152 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,152 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,152 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,152 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,152 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,152 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,153 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,152 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,153 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,153 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,153 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:05,153 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,153 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,153 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:05,153 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:05,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,153 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,154 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,154 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,154 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,154 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,154 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:05,154 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:05,254 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:05,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:05,277 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:05,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:05,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,280 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:05,280 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:05,281 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 00:54:05,281 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 00:54:05,281 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,281 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,282 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,282 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,282 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,283 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,284 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,284 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:05,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,284 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,284 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,285 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,285 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:05,285 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:05,286 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:05,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:05,287 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:05,287 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. started... 2023-02-11 00:54:05,287 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 00:54:05,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 00:54:05,287 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 00:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:05,288 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:05,288 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b0843d8ffc44a9678fe6589a79fd74e7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:05,289 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,291 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:05,306 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/103ed13f5ff849f28c10640a0411125e 2023-02-11 00:54:05,327 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/66bf8e5850d049b0891e2ebf67b738e6 2023-02-11 00:54:05,335 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/103ed13f5ff849f28c10640a0411125e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/103ed13f5ff849f28c10640a0411125e 2023-02-11 00:54:05,342 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/103ed13f5ff849f28c10640a0411125e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:54:05,343 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/66bf8e5850d049b0891e2ebf67b738e6 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/66bf8e5850d049b0891e2ebf67b738e6 2023-02-11 00:54:05,349 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/66bf8e5850d049b0891e2ebf67b738e6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:54:05,350 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b0843d8ffc44a9678fe6589a79fd74e7 in 62ms, sequenceid=71, compaction requested=true 2023-02-11 00:54:05,350 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b0843d8ffc44a9678fe6589a79fd74e7: 2023-02-11 00:54:05,351 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,352 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 00:54:05,352 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 00:54:05,354 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:05,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:05,354 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,354 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:05,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:05,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:05,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:05,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 00:54:05,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1db64e05[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:05,356 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:05,357 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,358 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:05,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:05,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:05,358 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:05,360 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,360 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:05,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:05,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:05,360 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:05,360 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:05,361 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:05,361 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 00:54:05,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:05,361 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:05,361 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:05,361 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:05,361 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 00:54:05,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:05,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:05,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:05,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:05,365 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 00:54:05,365 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:05,365 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:05,365 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,365 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:05,368 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:05,368 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:05,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,368 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,369 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:05,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,369 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:05,371 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,371 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,372 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,374 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,375 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:05,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:05,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:05,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 00:54:05,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,376 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:05,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:05,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:05,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:05,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:05,376 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:05,377 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:05,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:05,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:05,377 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,377 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,377 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 00:54:05,477 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:05,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:05,499 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,505 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:05,506 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:05,507 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:05,507 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:05,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b0843d8ffc44a9678fe6589a79fd74e7, UNASSIGN}] 2023-02-11 00:54:05,515 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b0843d8ffc44a9678fe6589a79fd74e7, UNASSIGN 2023-02-11 00:54:05,516 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=b0843d8ffc44a9678fe6589a79fd74e7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:05,516 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676076845516"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076845516"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076845516"}]},"ts":"1676076845516"} 2023-02-11 00:54:05,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure b0843d8ffc44a9678fe6589a79fd74e7, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:05,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b0843d8ffc44a9678fe6589a79fd74e7, disabling compactions & flushes 2023-02-11 00:54:05,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:05,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:05,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. after waiting 0 ms 2023-02-11 00:54:05,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:05,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing b0843d8ffc44a9678fe6589a79fd74e7 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 00:54:05,697 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a 2023-02-11 00:54:05,723 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/275c01a44dec4524ac2edccc19714760 2023-02-11 00:54:05,731 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a 2023-02-11 00:54:05,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:54:05,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.tmp/cf2/275c01a44dec4524ac2edccc19714760 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/275c01a44dec4524ac2edccc19714760 2023-02-11 00:54:05,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/275c01a44dec4524ac2edccc19714760, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:54:05,750 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b0843d8ffc44a9678fe6589a79fd74e7 in 71ms, sequenceid=82, compaction requested=true 2023-02-11 00:54:05,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 00:54:05,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:05,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. 2023-02-11 00:54:05,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b0843d8ffc44a9678fe6589a79fd74e7: Waiting for close lock at 1676076845678Running coprocessor pre-close hooks at 1676076845678Disabling compacts and flushes for region at 1676076845678Disabling writes for close at 1676076845678Obtaining lock to block concurrent updates at 1676076845678Preparing flush snapshotting stores in b0843d8ffc44a9678fe6589a79fd74e7 at 1676076845678Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676076845678Flushing stores of testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. at 1676076845679 (+1 ms)Flushing b0843d8ffc44a9678fe6589a79fd74e7/cf1: creating writer at 1676076845680 (+1 ms)Flushing b0843d8ffc44a9678fe6589a79fd74e7/cf1: appending metadata at 1676076845683 (+3 ms)Flushing b0843d8ffc44a9678fe6589a79fd74e7/cf1: closing flushed file at 1676076845683Flushing b0843d8ffc44a9678fe6589a79fd74e7/cf2: creating writer at 1676076845706 (+23 ms)Flushing b0843d8ffc44a9678fe6589a79fd74e7/cf2: appending metadata at 1676076845709 (+3 ms)Flushing b0843d8ffc44a9678fe6589a79fd74e7/cf2: closing flushed file at 1676076845709Flushing b0843d8ffc44a9678fe6589a79fd74e7/cf1: reopening flushed file at 1676076845732 (+23 ms)Flushing b0843d8ffc44a9678fe6589a79fd74e7/cf2: reopening flushed file at 1676076845743 (+11 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b0843d8ffc44a9678fe6589a79fd74e7 in 71ms, sequenceid=82, compaction requested=true at 1676076845750 (+7 ms)Writing region close event to WAL at 1676076845765 (+15 ms)Running coprocessor post-close hooks at 1676076845770 (+5 ms)Closed at 1676076845771 (+1 ms) 2023-02-11 00:54:05,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,775 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=b0843d8ffc44a9678fe6589a79fd74e7, regionState=CLOSED 2023-02-11 00:54:05,775 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676076845774"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076845774"}]},"ts":"1676076845774"} 2023-02-11 00:54:05,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-11 00:54:05,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure b0843d8ffc44a9678fe6589a79fd74e7, server=jenkins-hbase4.apache.org,38317,1676076840434 in 259 msec 2023-02-11 00:54:05,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-11 00:54:05,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b0843d8ffc44a9678fe6589a79fd74e7, UNASSIGN in 266 msec 2023-02-11 00:54:05,802 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=b0843d8ffc44a9678fe6589a79fd74e7, threads=14 2023-02-11 00:54:05,803 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/275c01a44dec4524ac2edccc19714760 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,803 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/66bf8e5850d049b0891e2ebf67b738e6 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,803 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,803 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,803 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,803 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,804 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/db42d0cbc71340bea870a75ad28dfcff for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,804 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/103ed13f5ff849f28c10640a0411125e for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,805 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,805 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/884619e6c3ea4ca4b866c5bf56209a53 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,805 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,806 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,806 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,806 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,833 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/103ed13f5ff849f28c10640a0411125e, top=true 2023-02-11 00:54:05,834 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/66bf8e5850d049b0891e2ebf67b738e6, top=true 2023-02-11 00:54:05,840 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/275c01a44dec4524ac2edccc19714760, top=true 2023-02-11 00:54:05,842 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/884619e6c3ea4ca4b866c5bf56209a53, top=true 2023-02-11 00:54:05,843 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/db42d0cbc71340bea870a75ad28dfcff, top=true 2023-02-11 00:54:05,849 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a, top=true 2023-02-11 00:54:05,915 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.splits/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-103ed13f5ff849f28c10640a0411125e for child: f6372215e78dd6cdebad6e3e039951b1, parent: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,915 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.splits/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-13dc23d1ed5c4f5eab6fde26ec14af4a for child: f6372215e78dd6cdebad6e3e039951b1, parent: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,915 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.splits/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-66bf8e5850d049b0891e2ebf67b738e6 for child: f6372215e78dd6cdebad6e3e039951b1, parent: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,915 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/66bf8e5850d049b0891e2ebf67b738e6 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,915 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/103ed13f5ff849f28c10640a0411125e for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,915 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,920 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.splits/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-275c01a44dec4524ac2edccc19714760 for child: f6372215e78dd6cdebad6e3e039951b1, parent: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,920 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.splits/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-884619e6c3ea4ca4b866c5bf56209a53 for child: f6372215e78dd6cdebad6e3e039951b1, parent: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,920 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/884619e6c3ea4ca4b866c5bf56209a53 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,920 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/275c01a44dec4524ac2edccc19714760 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,921 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/.splits/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-db42d0cbc71340bea870a75ad28dfcff for child: f6372215e78dd6cdebad6e3e039951b1, parent: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,924 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/db42d0cbc71340bea870a75ad28dfcff for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:05,998 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:06,002 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:06,002 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:06,012 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:06,026 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:06,026 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707 for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:06,028 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:06,369 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf for region: b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:06,369 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region b0843d8ffc44a9678fe6589a79fd74e7 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:54:06,422 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:54:06,424 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:54:06,426 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676076846426"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676076846426"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676076846426"}]},"ts":"1676076846426"} 2023-02-11 00:54:06,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076846426"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076846426"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076846426"}]},"ts":"1676076846426"} 2023-02-11 00:54:06,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076846426"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076846426"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076846426"}]},"ts":"1676076846426"} 2023-02-11 00:54:06,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=386584ef8660be854e06d144066a6e91, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f6372215e78dd6cdebad6e3e039951b1, ASSIGN}] 2023-02-11 00:54:06,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f6372215e78dd6cdebad6e3e039951b1, ASSIGN 2023-02-11 00:54:06,475 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f6372215e78dd6cdebad6e3e039951b1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:06,475 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=386584ef8660be854e06d144066a6e91, ASSIGN 2023-02-11 00:54:06,476 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=386584ef8660be854e06d144066a6e91, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:06,625 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 00:54:06,626 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=386584ef8660be854e06d144066a6e91, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:06,627 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076846626"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076846626"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076846626"}]},"ts":"1676076846626"} 2023-02-11 00:54:06,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 386584ef8660be854e06d144066a6e91, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:06,633 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=f6372215e78dd6cdebad6e3e039951b1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:06,633 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076846633"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076846633"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076846633"}]},"ts":"1676076846633"} 2023-02-11 00:54:06,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure f6372215e78dd6cdebad6e3e039951b1, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:06,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:06,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f6372215e78dd6cdebad6e3e039951b1, NAME => 'testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.', STARTKEY => '26', ENDKEY => ''} 2023-02-11 00:54:06,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:06,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:06,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:06,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:06,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:06,788 INFO [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:06,789 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1 2023-02-11 00:54:06,789 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1 2023-02-11 00:54:06,790 INFO [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f6372215e78dd6cdebad6e3e039951b1 columnFamilyName cf1 2023-02-11 00:54:06,823 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc-top 2023-02-11 00:54:06,831 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707-top 2023-02-11 00:54:06,839 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978-top 2023-02-11 00:54:06,848 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf-top 2023-02-11 00:54:06,854 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-103ed13f5ff849f28c10640a0411125e 2023-02-11 00:54:06,860 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-13dc23d1ed5c4f5eab6fde26ec14af4a 2023-02-11 00:54:06,865 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-884619e6c3ea4ca4b866c5bf56209a53 2023-02-11 00:54:06,866 INFO [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(310): Store=f6372215e78dd6cdebad6e3e039951b1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:06,866 INFO [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:06,867 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2 2023-02-11 00:54:06,867 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2 2023-02-11 00:54:06,868 INFO [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f6372215e78dd6cdebad6e3e039951b1 columnFamilyName cf2 2023-02-11 00:54:06,883 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b-top 2023-02-11 00:54:06,889 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c-top 2023-02-11 00:54:06,895 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04-top 2023-02-11 00:54:06,901 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7-top 2023-02-11 00:54:06,908 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-275c01a44dec4524ac2edccc19714760 2023-02-11 00:54:06,913 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-66bf8e5850d049b0891e2ebf67b738e6 2023-02-11 00:54:06,918 DEBUG [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-db42d0cbc71340bea870a75ad28dfcff 2023-02-11 00:54:06,918 INFO [StoreOpener-f6372215e78dd6cdebad6e3e039951b1-1] regionserver.HStore(310): Store=f6372215e78dd6cdebad6e3e039951b1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:06,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:06,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:06,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:06,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:06,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f6372215e78dd6cdebad6e3e039951b1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71573721, jitterRate=0.06653155386447906}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:06,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f6372215e78dd6cdebad6e3e039951b1: Running coprocessor pre-open hook at 1676076846787Writing region info on filesystem at 1676076846787Initializing all the Stores at 1676076846788 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076846788Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076846788Cleaning up temporary data from old regions at 1676076846923 (+135 ms)Cleaning up detritus from prior splits at 1676076846923Running coprocessor post-open hooks at 1676076846926 (+3 ms)Region opened successfully at 1676076846926 2023-02-11 00:54:06,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1., pid=18, masterSystemTime=1676076846781 2023-02-11 00:54:06,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:06,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:54:06,931 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:06,931 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:06,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:06,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:06,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:06,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 386584ef8660be854e06d144066a6e91, NAME => 'testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.', STARTKEY => '', ENDKEY => '26'} 2023-02-11 00:54:06,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:06,934 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=f6372215e78dd6cdebad6e3e039951b1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:06,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 386584ef8660be854e06d144066a6e91 2023-02-11 00:54:06,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:06,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 386584ef8660be854e06d144066a6e91 2023-02-11 00:54:06,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 386584ef8660be854e06d144066a6e91 2023-02-11 00:54:06,934 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076846934"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076846934"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076846934"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076846934"}]},"ts":"1676076846934"} 2023-02-11 00:54:06,935 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:06,935 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1912): f6372215e78dd6cdebad6e3e039951b1/cf1 is initiating minor compaction (all files) 2023-02-11 00:54:06,935 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:06,936 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f6372215e78dd6cdebad6e3e039951b1/cf1 in testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:06,936 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1912): f6372215e78dd6cdebad6e3e039951b1/cf2 is initiating minor compaction (all files) 2023-02-11 00:54:06,937 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f6372215e78dd6cdebad6e3e039951b1/cf2 in testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:06,937 INFO [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 386584ef8660be854e06d144066a6e91 2023-02-11 00:54:06,937 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-884619e6c3ea4ca4b866c5bf56209a53, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-103ed13f5ff849f28c10640a0411125e, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-13dc23d1ed5c4f5eab6fde26ec14af4a] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/.tmp, totalSize=34.9 K 2023-02-11 00:54:06,937 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-db42d0cbc71340bea870a75ad28dfcff, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-66bf8e5850d049b0891e2ebf67b738e6, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-275c01a44dec4524ac2edccc19714760] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/.tmp, totalSize=34.9 K 2023-02-11 00:54:06,939 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076843912 2023-02-11 00:54:06,939 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076843912 2023-02-11 00:54:06,940 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1 2023-02-11 00:54:06,940 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1 2023-02-11 00:54:06,941 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076844280 2023-02-11 00:54:06,941 INFO [StoreOpener-386584ef8660be854e06d144066a6e91-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 386584ef8660be854e06d144066a6e91 columnFamilyName cf1 2023-02-11 00:54:06,941 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076844280 2023-02-11 00:54:06,942 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076844544 2023-02-11 00:54:06,943 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076844544 2023-02-11 00:54:06,943 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676076844777 2023-02-11 00:54:06,945 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676076844777 2023-02-11 00:54:06,945 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-db42d0cbc71340bea870a75ad28dfcff, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076845019 2023-02-11 00:54:06,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-11 00:54:06,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure f6372215e78dd6cdebad6e3e039951b1, server=jenkins-hbase4.apache.org,38317,1676076840434 in 304 msec 2023-02-11 00:54:06,945 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-884619e6c3ea4ca4b866c5bf56209a53, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076845019 2023-02-11 00:54:06,946 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-103ed13f5ff849f28c10640a0411125e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076845257 2023-02-11 00:54:06,946 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-66bf8e5850d049b0891e2ebf67b738e6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076845257 2023-02-11 00:54:06,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f6372215e78dd6cdebad6e3e039951b1, ASSIGN in 472 msec 2023-02-11 00:54:06,947 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-13dc23d1ed5c4f5eab6fde26ec14af4a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076845479 2023-02-11 00:54:06,947 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-275c01a44dec4524ac2edccc19714760, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076845479 2023-02-11 00:54:06,966 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc-bottom 2023-02-11 00:54:06,971 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707-bottom 2023-02-11 00:54:06,981 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978-bottom 2023-02-11 00:54:06,983 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] throttle.PressureAwareThroughputController(145): f6372215e78dd6cdebad6e3e039951b1#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:06,983 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] throttle.PressureAwareThroughputController(145): f6372215e78dd6cdebad6e3e039951b1#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:06,993 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf-bottom 2023-02-11 00:54:06,996 INFO [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(310): Store=386584ef8660be854e06d144066a6e91/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:06,996 INFO [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 386584ef8660be854e06d144066a6e91 2023-02-11 00:54:06,998 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2 2023-02-11 00:54:06,998 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2 2023-02-11 00:54:06,999 INFO [StoreOpener-386584ef8660be854e06d144066a6e91-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 386584ef8660be854e06d144066a6e91 columnFamilyName cf2 2023-02-11 00:54:07,030 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/.tmp/cf2/742ee1594e254b7a9d7ab559f211aed2 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/742ee1594e254b7a9d7ab559f211aed2 2023-02-11 00:54:07,040 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/.tmp/cf1/29e63d6b7d5344b9b26292c3255ad8bf as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/29e63d6b7d5344b9b26292c3255ad8bf 2023-02-11 00:54:07,041 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b-bottom 2023-02-11 00:54:07,050 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c-bottom 2023-02-11 00:54:07,058 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04-bottom 2023-02-11 00:54:07,062 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f6372215e78dd6cdebad6e3e039951b1/cf1 of f6372215e78dd6cdebad6e3e039951b1 into 29e63d6b7d5344b9b26292c3255ad8bf(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:07,063 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f6372215e78dd6cdebad6e3e039951b1/cf2 of f6372215e78dd6cdebad6e3e039951b1 into 742ee1594e254b7a9d7ab559f211aed2(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:07,064 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f6372215e78dd6cdebad6e3e039951b1: 2023-02-11 00:54:07,064 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f6372215e78dd6cdebad6e3e039951b1: 2023-02-11 00:54:07,064 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1., storeName=f6372215e78dd6cdebad6e3e039951b1/cf2, priority=9, startTime=1676076846929; duration=0sec 2023-02-11 00:54:07,064 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1., storeName=f6372215e78dd6cdebad6e3e039951b1/cf1, priority=9, startTime=1676076846929; duration=0sec 2023-02-11 00:54:07,065 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:07,065 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:07,066 DEBUG [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7-bottom 2023-02-11 00:54:07,067 INFO [StoreOpener-386584ef8660be854e06d144066a6e91-1] regionserver.HStore(310): Store=386584ef8660be854e06d144066a6e91/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:07,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91 2023-02-11 00:54:07,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91 2023-02-11 00:54:07,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:07,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 386584ef8660be854e06d144066a6e91 2023-02-11 00:54:07,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 386584ef8660be854e06d144066a6e91; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66037279, jitterRate=-0.01596786081790924}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:07,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 386584ef8660be854e06d144066a6e91: Running coprocessor pre-open hook at 1676076846935Writing region info on filesystem at 1676076846935Initializing all the Stores at 1676076846937 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076846937Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076846937Cleaning up temporary data from old regions at 1676076847070 (+133 ms)Cleaning up detritus from prior splits at 1676076847071 (+1 ms)Running coprocessor post-open hooks at 1676076847075 (+4 ms)Region opened successfully at 1676076847075 2023-02-11 00:54:07,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91., pid=17, masterSystemTime=1676076846781 2023-02-11 00:54:07,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:07,077 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 00:54:07,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:07,079 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 00:54:07,080 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,080 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1912): 386584ef8660be854e06d144066a6e91/cf1 is initiating minor compaction (all files) 2023-02-11 00:54:07,080 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,080 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1912): 386584ef8660be854e06d144066a6e91/cf2 is initiating minor compaction (all files) 2023-02-11 00:54:07,080 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 386584ef8660be854e06d144066a6e91/cf2 in testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,080 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 386584ef8660be854e06d144066a6e91/cf1 in testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,081 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/.tmp, totalSize=19.9 K 2023-02-11 00:54:07,081 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/.tmp, totalSize=19.9 K 2023-02-11 00:54:07,082 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076843912 2023-02-11 00:54:07,083 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076844280 2023-02-11 00:54:07,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,083 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076844544 2023-02-11 00:54:07,083 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076843912 2023-02-11 00:54:07,084 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076844777 2023-02-11 00:54:07,084 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=386584ef8660be854e06d144066a6e91, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:07,084 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076844280 2023-02-11 00:54:07,084 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076847084"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076847084"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076847084"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076847084"}]},"ts":"1676076847084"} 2023-02-11 00:54:07,086 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076844544 2023-02-11 00:54:07,088 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076844777 2023-02-11 00:54:07,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-11 00:54:07,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 386584ef8660be854e06d144066a6e91, server=jenkins-hbase4.apache.org,38317,1676076840434 in 458 msec 2023-02-11 00:54:07,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-11 00:54:07,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=386584ef8660be854e06d144066a6e91, ASSIGN in 621 msec 2023-02-11 00:54:07,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 in 1.5990 sec 2023-02-11 00:54:07,112 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] throttle.PressureAwareThroughputController(145): 386584ef8660be854e06d144066a6e91#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:07,112 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] throttle.PressureAwareThroughputController(145): 386584ef8660be854e06d144066a6e91#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:07,135 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/.tmp/cf1/7440d22a33684b66a4e14427d3442fd5 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/7440d22a33684b66a4e14427d3442fd5 2023-02-11 00:54:07,137 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/.tmp/cf2/d65f231df22343f48f21986c4942cbdf as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/d65f231df22343f48f21986c4942cbdf 2023-02-11 00:54:07,142 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 386584ef8660be854e06d144066a6e91/cf1 of 386584ef8660be854e06d144066a6e91 into 7440d22a33684b66a4e14427d3442fd5(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:07,143 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 386584ef8660be854e06d144066a6e91: 2023-02-11 00:54:07,143 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91., storeName=386584ef8660be854e06d144066a6e91/cf1, priority=12, startTime=1676076847076; duration=0sec 2023-02-11 00:54:07,143 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:07,145 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 386584ef8660be854e06d144066a6e91/cf2 of 386584ef8660be854e06d144066a6e91 into d65f231df22343f48f21986c4942cbdf(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:07,145 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 386584ef8660be854e06d144066a6e91: 2023-02-11 00:54:07,145 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91., storeName=386584ef8660be854e06d144066a6e91/cf2, priority=12, startTime=1676076847077; duration=0sec 2023-02-11 00:54:07,145 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:07,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38317] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:45472 deadline: 1676076907258, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7. is not online on jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:07,375 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-11 00:54:07,386 INFO [Listener at localhost/35701] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 00:54:07,389 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-11 00:54:07,420 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:07,430 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,436 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076847447"}]},"ts":"1676076847447"} 2023-02-11 00:54:07,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-11 00:54:07,451 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-11 00:54:07,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 00:54:07,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=386584ef8660be854e06d144066a6e91, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f6372215e78dd6cdebad6e3e039951b1, UNASSIGN}] 2023-02-11 00:54:07,455 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f6372215e78dd6cdebad6e3e039951b1, UNASSIGN 2023-02-11 00:54:07,455 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=f6372215e78dd6cdebad6e3e039951b1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:07,456 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076847455"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076847455"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076847455"}]},"ts":"1676076847455"} 2023-02-11 00:54:07,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure f6372215e78dd6cdebad6e3e039951b1, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:07,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=386584ef8660be854e06d144066a6e91, UNASSIGN 2023-02-11 00:54:07,463 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=386584ef8660be854e06d144066a6e91, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:07,463 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076847463"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076847463"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076847463"}]},"ts":"1676076847463"} 2023-02-11 00:54:07,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 386584ef8660be854e06d144066a6e91, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:07,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 00:54:07,611 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 386584ef8660be854e06d144066a6e91 2023-02-11 00:54:07,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 386584ef8660be854e06d144066a6e91, disabling compactions & flushes 2023-02-11 00:54:07,611 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. after waiting 0 ms 2023-02-11 00:54:07,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,617 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707-bottom] to archive 2023-02-11 00:54:07,619 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:07,624 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,627 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,629 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,631 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,657 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c-bottom] to archive 2023-02-11 00:54:07,658 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:07,660 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,662 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,664 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,665 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:54:07,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:07,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91. 2023-02-11 00:54:07,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 386584ef8660be854e06d144066a6e91: Waiting for close lock at 1676076847611Running coprocessor pre-close hooks at 1676076847611Disabling compacts and flushes for region at 1676076847611Disabling writes for close at 1676076847611Writing region close event to WAL at 1676076847666 (+55 ms)Running coprocessor post-close hooks at 1676076847672 (+6 ms)Closed at 1676076847672 2023-02-11 00:54:07,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 386584ef8660be854e06d144066a6e91 2023-02-11 00:54:07,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:07,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f6372215e78dd6cdebad6e3e039951b1, disabling compactions & flushes 2023-02-11 00:54:07,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:07,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:07,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. after waiting 0 ms 2023-02-11 00:54:07,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:07,675 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=386584ef8660be854e06d144066a6e91, regionState=CLOSED 2023-02-11 00:54:07,675 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076847675"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076847675"}]},"ts":"1676076847675"} 2023-02-11 00:54:07,676 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-884619e6c3ea4ca4b866c5bf56209a53, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-103ed13f5ff849f28c10640a0411125e, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-13dc23d1ed5c4f5eab6fde26ec14af4a] to archive 2023-02-11 00:54:07,677 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:07,679 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/907055a24ff04fb7a5aa8f20cbf36bbc.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-11 00:54:07,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 386584ef8660be854e06d144066a6e91, server=jenkins-hbase4.apache.org,38317,1676076840434 in 212 msec 2023-02-11 00:54:07,681 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/e75819f609ce44fbbe4affb942bdc978.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=386584ef8660be854e06d144066a6e91, UNASSIGN in 228 msec 2023-02-11 00:54:07,683 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/ff24e1efe2e444d28c962ad988232caf.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,685 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/abfb1b588828418cbd77c456c4781707.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,686 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-884619e6c3ea4ca4b866c5bf56209a53 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-884619e6c3ea4ca4b866c5bf56209a53 2023-02-11 00:54:07,687 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-103ed13f5ff849f28c10640a0411125e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-103ed13f5ff849f28c10640a0411125e 2023-02-11 00:54:07,689 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-13dc23d1ed5c4f5eab6fde26ec14af4a to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-13dc23d1ed5c4f5eab6fde26ec14af4a 2023-02-11 00:54:07,691 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-db42d0cbc71340bea870a75ad28dfcff, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-66bf8e5850d049b0891e2ebf67b738e6, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-275c01a44dec4524ac2edccc19714760] to archive 2023-02-11 00:54:07,693 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:07,695 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c2d4c491a3e74530888035bb12adfb04.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,696 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/67f05b137d6543089693339e96d9746b.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,697 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/c9f3035466c04d68a5923bdffcc19eb7.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,699 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/8aad1841ef1040e68a6caea9bf21b53c.b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,701 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-db42d0cbc71340bea870a75ad28dfcff to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-db42d0cbc71340bea870a75ad28dfcff 2023-02-11 00:54:07,702 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-66bf8e5850d049b0891e2ebf67b738e6 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-66bf8e5850d049b0891e2ebf67b738e6 2023-02-11 00:54:07,704 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-275c01a44dec4524ac2edccc19714760 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/testSplitTableRegionUnevenDaughter=b0843d8ffc44a9678fe6589a79fd74e7-275c01a44dec4524ac2edccc19714760 2023-02-11 00:54:07,710 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:54:07,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:07,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1. 2023-02-11 00:54:07,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f6372215e78dd6cdebad6e3e039951b1: Waiting for close lock at 1676076847674Running coprocessor pre-close hooks at 1676076847674Disabling compacts and flushes for region at 1676076847674Disabling writes for close at 1676076847674Writing region close event to WAL at 1676076847705 (+31 ms)Running coprocessor post-close hooks at 1676076847711 (+6 ms)Closed at 1676076847712 (+1 ms) 2023-02-11 00:54:07,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:07,714 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=f6372215e78dd6cdebad6e3e039951b1, regionState=CLOSED 2023-02-11 00:54:07,715 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676076847714"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076847714"}]},"ts":"1676076847714"} 2023-02-11 00:54:07,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-11 00:54:07,720 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure f6372215e78dd6cdebad6e3e039951b1, server=jenkins-hbase4.apache.org,38317,1676076840434 in 259 msec 2023-02-11 00:54:07,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-11 00:54:07,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f6372215e78dd6cdebad6e3e039951b1, UNASSIGN in 267 msec 2023-02-11 00:54:07,724 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076847724"}]},"ts":"1676076847724"} 2023-02-11 00:54:07,726 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-11 00:54:07,728 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-11 00:54:07,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 292 msec 2023-02-11 00:54:07,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 00:54:07,760 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-11 00:54:07,766 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,775 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,777 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 00:54:07,786 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,786 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91 2023-02-11 00:54:07,786 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:07,791 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/recovered.edits] 2023-02-11 00:54:07,791 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/recovered.edits] 2023-02-11 00:54:07,791 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/recovered.edits] 2023-02-11 00:54:07,795 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 00:54:07,797 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/103ed13f5ff849f28c10640a0411125e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/103ed13f5ff849f28c10640a0411125e 2023-02-11 00:54:07,797 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/7440d22a33684b66a4e14427d3442fd5 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf1/7440d22a33684b66a4e14427d3442fd5 2023-02-11 00:54:07,799 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/13dc23d1ed5c4f5eab6fde26ec14af4a 2023-02-11 00:54:07,799 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/29e63d6b7d5344b9b26292c3255ad8bf to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf1/29e63d6b7d5344b9b26292c3255ad8bf 2023-02-11 00:54:07,803 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/884619e6c3ea4ca4b866c5bf56209a53 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/884619e6c3ea4ca4b866c5bf56209a53 2023-02-11 00:54:07,804 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/d65f231df22343f48f21986c4942cbdf to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/cf2/d65f231df22343f48f21986c4942cbdf 2023-02-11 00:54:07,806 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/907055a24ff04fb7a5aa8f20cbf36bbc 2023-02-11 00:54:07,814 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/742ee1594e254b7a9d7ab559f211aed2 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/cf2/742ee1594e254b7a9d7ab559f211aed2 2023-02-11 00:54:07,816 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/abfb1b588828418cbd77c456c4781707 2023-02-11 00:54:07,820 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91/recovered.edits/90.seqid 2023-02-11 00:54:07,820 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/e75819f609ce44fbbe4affb942bdc978 2023-02-11 00:54:07,821 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1/recovered.edits/90.seqid 2023-02-11 00:54:07,822 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/386584ef8660be854e06d144066a6e91 2023-02-11 00:54:07,822 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:07,823 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf1/ff24e1efe2e444d28c962ad988232caf 2023-02-11 00:54:07,826 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/275c01a44dec4524ac2edccc19714760 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/275c01a44dec4524ac2edccc19714760 2023-02-11 00:54:07,828 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/66bf8e5850d049b0891e2ebf67b738e6 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/66bf8e5850d049b0891e2ebf67b738e6 2023-02-11 00:54:07,829 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/67f05b137d6543089693339e96d9746b 2023-02-11 00:54:07,831 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/8aad1841ef1040e68a6caea9bf21b53c 2023-02-11 00:54:07,833 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c2d4c491a3e74530888035bb12adfb04 2023-02-11 00:54:07,834 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/c9f3035466c04d68a5923bdffcc19eb7 2023-02-11 00:54:07,836 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/db42d0cbc71340bea870a75ad28dfcff to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/cf2/db42d0cbc71340bea870a75ad28dfcff 2023-02-11 00:54:07,840 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/recovered.edits/85.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7/recovered.edits/85.seqid 2023-02-11 00:54:07,841 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionUnevenDaughter/b0843d8ffc44a9678fe6589a79fd74e7 2023-02-11 00:54:07,841 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-11 00:54:07,844 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,874 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-11 00:54:07,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 00:54:07,885 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-11 00:54:07,887 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-11 00:54:07,919 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-11 00:54:07,921 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,921 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-11 00:54:07,921 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076847921"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:07,921 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076847921"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:07,921 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076847921"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:07,924 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 00:54:07,924 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b0843d8ffc44a9678fe6589a79fd74e7, NAME => 'testSplitTableRegionUnevenDaughter,,1676076843401.b0843d8ffc44a9678fe6589a79fd74e7.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 386584ef8660be854e06d144066a6e91, NAME => 'testSplitTableRegionUnevenDaughter,,1676076845499.386584ef8660be854e06d144066a6e91.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => f6372215e78dd6cdebad6e3e039951b1, NAME => 'testSplitTableRegionUnevenDaughter,26,1676076845499.f6372215e78dd6cdebad6e3e039951b1.', STARTKEY => '26', ENDKEY => ''}] 2023-02-11 00:54:07,924 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-11 00:54:07,925 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076847925"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:07,927 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-11 00:54:07,930 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:07,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 163 msec 2023-02-11 00:54:08,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 00:54:08,082 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-11 00:54:08,099 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=395 (was 363) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:48060 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:49918 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79353ebb-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:49906 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79353ebb-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79353ebb-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:38317-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47762 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:50748 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41704 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:49866 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47824 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47716 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47856 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41416 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41692 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41720 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79353ebb-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47806 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41488 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47692 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=835 (was 700) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 247), ProcessCount=168 (was 168), AvailableMemoryMB=3413 (was 3582) 2023-02-11 00:54:08,114 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=395, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=168, AvailableMemoryMB=3413 2023-02-11 00:54:08,114 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:08,119 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:54:08,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:54:08,121 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:08,123 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:54:08,124 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:54:08,126 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,127 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613 empty. 2023-02-11 00:54:08,127 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,127 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-11 00:54:08,139 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:08,140 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => a4578dcd4830de76f0230a8be7356613, NAME => 'testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:54:08,152 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:08,152 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing a4578dcd4830de76f0230a8be7356613, disabling compactions & flushes 2023-02-11 00:54:08,152 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,152 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,152 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. after waiting 0 ms 2023-02-11 00:54:08,152 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,152 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,152 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for a4578dcd4830de76f0230a8be7356613: Waiting for close lock at 1676076848152Disabling compacts and flushes for region at 1676076848152Disabling writes for close at 1676076848152Writing region close event to WAL at 1676076848152Closed at 1676076848152 2023-02-11 00:54:08,155 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:54:08,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076848156"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076848156"}]},"ts":"1676076848156"} 2023-02-11 00:54:08,158 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:54:08,159 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:54:08,160 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076848159"}]},"ts":"1676076848159"} 2023-02-11 00:54:08,161 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-11 00:54:08,165 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:08,166 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:08,166 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:08,166 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:08,167 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:54:08,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, ASSIGN}] 2023-02-11 00:54:08,169 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, ASSIGN 2023-02-11 00:54:08,169 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:08,320 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:08,321 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=a4578dcd4830de76f0230a8be7356613, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,321 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076848321"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076848321"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076848321"}]},"ts":"1676076848321"} 2023-02-11 00:54:08,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:08,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a4578dcd4830de76f0230a8be7356613, NAME => 'testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:08,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:08,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:08,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,481 INFO [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,483 DEBUG [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1 2023-02-11 00:54:08,483 DEBUG [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1 2023-02-11 00:54:08,483 INFO [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4578dcd4830de76f0230a8be7356613 columnFamilyName cf1 2023-02-11 00:54:08,484 INFO [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] regionserver.HStore(310): Store=a4578dcd4830de76f0230a8be7356613/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:08,484 INFO [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,486 DEBUG [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2 2023-02-11 00:54:08,486 DEBUG [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2 2023-02-11 00:54:08,487 INFO [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4578dcd4830de76f0230a8be7356613 columnFamilyName cf2 2023-02-11 00:54:08,488 INFO [StoreOpener-a4578dcd4830de76f0230a8be7356613-1] regionserver.HStore(310): Store=a4578dcd4830de76f0230a8be7356613/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:08,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:08,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:08,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:08,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a4578dcd4830de76f0230a8be7356613; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60791344, jitterRate=-0.09413838386535645}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:08,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a4578dcd4830de76f0230a8be7356613: Running coprocessor pre-open hook at 1676076848479Writing region info on filesystem at 1676076848480 (+1 ms)Initializing all the Stores at 1676076848480Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076848480Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076848481 (+1 ms)Cleaning up temporary data from old regions at 1676076848490 (+9 ms)Cleaning up detritus from prior splits at 1676076848491 (+1 ms)Running coprocessor post-open hooks at 1676076848496 (+5 ms)Region opened successfully at 1676076848496 2023-02-11 00:54:08,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613., pid=27, masterSystemTime=1676076848475 2023-02-11 00:54:08,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,500 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=a4578dcd4830de76f0230a8be7356613, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,500 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076848499"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076848499"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076848499"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076848499"}]},"ts":"1676076848499"} 2023-02-11 00:54:08,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-11 00:54:08,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 in 179 msec 2023-02-11 00:54:08,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-11 00:54:08,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, ASSIGN in 337 msec 2023-02-11 00:54:08,508 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:54:08,508 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076848508"}]},"ts":"1676076848508"} 2023-02-11 00:54:08,509 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-11 00:54:08,512 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:54:08,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 392 msec 2023-02-11 00:54:08,633 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:08,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 00:54:08,635 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 00:54:08,635 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:08,636 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 00:54:08,636 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 00:54:08,636 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,637 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,638 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,638 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,638 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,638 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,638 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,639 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:08,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:08,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,639 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:08,640 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:08,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:08,640 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:08,640 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:08,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:08,644 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:08,644 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:08,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:08,645 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:08,645 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,645 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:08,645 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:08,645 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:08,644 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:08,645 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:08,645 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. started... 2023-02-11 00:54:08,646 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:08,646 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:08,646 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4578dcd4830de76f0230a8be7356613 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 00:54:08,646 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:08,647 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:08,648 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,651 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:08,659 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/ba378566ea874463b18342836eb906ef 2023-02-11 00:54:08,674 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/1551b7d98f8b44f5b3df7284af3b53b0 2023-02-11 00:54:08,679 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/ba378566ea874463b18342836eb906ef as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef 2023-02-11 00:54:08,684 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:54:08,684 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/1551b7d98f8b44f5b3df7284af3b53b0 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0 2023-02-11 00:54:08,689 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:54:08,690 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a4578dcd4830de76f0230a8be7356613 in 44ms, sequenceid=6, compaction requested=false 2023-02-11 00:54:08,691 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-11 00:54:08,692 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4578dcd4830de76f0230a8be7356613: 2023-02-11 00:54:08,692 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:08,694 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:08,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:08,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:08,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:08,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 00:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2044517d[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:08,697 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 00:54:08,697 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:08,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:08,698 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,698 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:08,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:08,699 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:08,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,699 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:08,699 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:08,699 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:08,699 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:08,699 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:08,700 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,700 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:08,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:08,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:08,700 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:08,701 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:08,702 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:08,702 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:08,702 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:08,702 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:08,702 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:08,703 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:08,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:08,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:08,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:08,706 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 00:54:08,707 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 00:54:08,707 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:08,707 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:08,707 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 00:54:08,707 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:08,708 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,708 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:08,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:08,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,708 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,708 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,708 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:08,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,709 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:08,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,710 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,710 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:08,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,712 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,712 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,713 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,715 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,716 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 00:54:08,716 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:08,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,716 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:08,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,717 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:08,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,717 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,717 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:08,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,717 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,718 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,718 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,718 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,718 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,818 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:08,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:08,836 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:08,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:08,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 00:54:08,838 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 00:54:08,838 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:08,839 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 00:54:08,839 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 00:54:08,839 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,839 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,840 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,840 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,840 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,840 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,840 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,841 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,841 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,841 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:08,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:08,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:08,842 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:08,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:08,842 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:08,843 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:08,843 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:08,843 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:08,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:08,844 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:08,844 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:08,844 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:08,844 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:08,844 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:08,844 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:08,844 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:08,845 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:08,844 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,845 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:08,845 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:08,845 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. started... 2023-02-11 00:54:08,845 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4578dcd4830de76f0230a8be7356613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:08,847 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,847 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,848 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:08,848 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,849 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,849 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:08,858 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/3d9972b749ae4f1d840591ba0515b3ed 2023-02-11 00:54:08,872 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 2023-02-11 00:54:08,878 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/3d9972b749ae4f1d840591ba0515b3ed as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed 2023-02-11 00:54:08,882 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:54:08,883 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 2023-02-11 00:54:08,888 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:54:08,889 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4578dcd4830de76f0230a8be7356613 in 44ms, sequenceid=19, compaction requested=false 2023-02-11 00:54:08,889 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4578dcd4830de76f0230a8be7356613: 2023-02-11 00:54:08,889 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:08,889 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:08,891 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,891 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:08,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:08,891 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,891 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:08,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:08,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:08,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:08,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 00:54:08,894 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2caf63e3[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:08,894 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 00:54:08,894 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:08,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:08,896 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,896 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:08,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:08,897 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:08,897 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:08,896 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:08,899 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:08,899 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:08,899 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:08,907 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:08,907 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,907 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:08,907 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:08,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:08,908 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:08,908 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:08,908 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:08,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:08,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:08,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:08,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:08,913 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 00:54:08,913 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 00:54:08,913 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:08,913 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:08,913 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 00:54:08,913 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:08,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,915 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,916 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,916 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,916 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:08,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,917 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,918 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:08,918 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:08,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,921 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:08,921 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,921 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:08,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,925 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,925 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,925 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,925 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,925 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,925 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,926 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 00:54:08,926 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:08,925 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:08,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,926 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:08,926 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:08,926 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,926 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:08,926 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:08,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,926 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:08,926 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:08,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:08,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:08,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:08,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:08,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:08,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:08,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:08,928 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,027 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:09,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:09,048 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:09,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:09,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 00:54:09,050 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 00:54:09,050 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,051 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 00:54:09,051 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 00:54:09,051 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,051 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,053 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,053 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,053 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,053 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,053 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,053 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,055 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,055 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,055 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,055 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,056 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,056 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,055 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,057 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:09,057 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. started... 2023-02-11 00:54:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,057 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4578dcd4830de76f0230a8be7356613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:09,057 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,059 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,060 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,060 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,060 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,060 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,060 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,070 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/7471952cfabc4abea2f5c2fa96ce09ed 2023-02-11 00:54:09,085 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e 2023-02-11 00:54:09,090 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/7471952cfabc4abea2f5c2fa96ce09ed as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed 2023-02-11 00:54:09,095 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:54:09,096 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e 2023-02-11 00:54:09,101 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:54:09,102 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4578dcd4830de76f0230a8be7356613 in 45ms, sequenceid=32, compaction requested=true 2023-02-11 00:54:09,102 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4578dcd4830de76f0230a8be7356613: 2023-02-11 00:54:09,102 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:09,102 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:09,102 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:09,102 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,102 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,102 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,103 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,103 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,104 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,104 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 00:54:09,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65291fb[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:09,106 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 00:54:09,106 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,107 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,107 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:09,107 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:09,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:09,107 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:09,109 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,109 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,109 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,109 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:09,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 00:54:09,115 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 00:54:09,115 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:09,115 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:09,115 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 00:54:09,115 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:09,116 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,116 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,116 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,117 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,116 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,116 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,117 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,117 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,118 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,119 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,121 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,121 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,121 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,124 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,124 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,124 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,124 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,124 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,124 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,125 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,125 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,124 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,125 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,125 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,125 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,125 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 00:54:09,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,125 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,126 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,126 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,126 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,126 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,126 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,126 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,126 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:09,127 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:09,227 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:09,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:09,247 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:09,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:09,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 00:54:09,248 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 00:54:09,249 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,249 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 00:54:09,249 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 00:54:09,250 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,250 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,251 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,251 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,251 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,251 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,252 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,252 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,252 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,252 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,252 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,255 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,255 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,255 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,255 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,255 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,256 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,256 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,256 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,256 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,256 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,256 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,257 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,258 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,258 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,258 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,258 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:09,258 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:09,258 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:09,259 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. started... 2023-02-11 00:54:09,259 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4578dcd4830de76f0230a8be7356613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:09,263 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,264 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,264 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,264 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,264 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,264 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,270 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/3155732a33c14a41a0a436742c7bc1da 2023-02-11 00:54:09,286 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/fb2d53d58cbe489ca60e05194f85f1dd 2023-02-11 00:54:09,291 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/3155732a33c14a41a0a436742c7bc1da as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da 2023-02-11 00:54:09,296 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:54:09,297 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/fb2d53d58cbe489ca60e05194f85f1dd as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd 2023-02-11 00:54:09,302 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:54:09,303 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4578dcd4830de76f0230a8be7356613 in 44ms, sequenceid=45, compaction requested=true 2023-02-11 00:54:09,303 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4578dcd4830de76f0230a8be7356613: 2023-02-11 00:54:09,303 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:09,303 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:09,303 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:09,303 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,303 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,303 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,305 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,305 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,305 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,305 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 00:54:09,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29bb2152[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:09,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 00:54:09,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,308 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,308 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,308 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:09,309 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,309 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:09,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,309 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:09,311 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,311 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,311 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,312 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,313 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,313 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,313 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,313 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,313 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,314 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:09,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 00:54:09,318 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 00:54:09,318 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:09,318 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:09,318 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 00:54:09,318 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:09,320 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,320 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,320 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,320 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,320 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,321 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,321 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,323 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,323 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,323 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,326 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,326 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,326 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,327 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,326 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,326 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,327 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 00:54:09,327 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:09,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,328 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:09,328 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:09,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,327 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,328 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,328 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,328 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,328 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,387 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 00:54:09,388 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-11 00:54:09,388 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:09,388 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-11 00:54:09,428 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:09,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:09,448 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:09,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:09,450 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 00:54:09,450 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 00:54:09,450 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,450 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 00:54:09,450 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 00:54:09,450 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,451 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,453 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,453 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,453 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,453 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,454 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,454 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,455 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:09,456 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:09,456 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. started... 2023-02-11 00:54:09,456 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4578dcd4830de76f0230a8be7356613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:09,457 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,458 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,458 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,458 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,458 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,458 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,470 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/cb7632271f904b0680638b360a4334f3 2023-02-11 00:54:09,491 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/88c00ba48f834f33bcd361eacc97762f 2023-02-11 00:54:09,497 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/cb7632271f904b0680638b360a4334f3 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3 2023-02-11 00:54:09,502 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:54:09,503 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/88c00ba48f834f33bcd361eacc97762f as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f 2023-02-11 00:54:09,508 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:54:09,509 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4578dcd4830de76f0230a8be7356613 in 53ms, sequenceid=58, compaction requested=true 2023-02-11 00:54:09,509 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4578dcd4830de76f0230a8be7356613: 2023-02-11 00:54:09,509 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:09,509 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:09,509 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:09,509 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,509 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,509 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,511 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,511 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,511 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,511 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 00:54:09,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b7642a1[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:09,513 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 00:54:09,513 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,515 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,516 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:09,515 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,515 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,516 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,516 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,516 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:09,516 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:09,516 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,516 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:09,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,517 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,517 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,517 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,518 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,518 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,518 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,518 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,518 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,518 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:09,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 00:54:09,521 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 00:54:09,521 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:09,521 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:09,521 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 00:54:09,521 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:09,522 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,522 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,522 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,522 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,523 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,523 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,524 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,525 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,526 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,526 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,532 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,532 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 00:54:09,532 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,533 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:09,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,533 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:09,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,633 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:09,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:09,651 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:09,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:09,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 00:54:09,653 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 00:54:09,653 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,653 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 00:54:09,653 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 00:54:09,653 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,653 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,655 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,655 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,655 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 00:54:09,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,656 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,657 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:09,657 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:09,657 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:09,657 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. started... 2023-02-11 00:54:09,657 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a4578dcd4830de76f0230a8be7356613 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:09,658 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,658 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,659 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,659 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,659 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,659 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,668 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/c68fa8a7cb3541faa8fa033756bb1a2e 2023-02-11 00:54:09,681 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/90466a24df1d4ca2b8b7a8b112578255 2023-02-11 00:54:09,686 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/c68fa8a7cb3541faa8fa033756bb1a2e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e 2023-02-11 00:54:09,692 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:54:09,693 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/90466a24df1d4ca2b8b7a8b112578255 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255 2023-02-11 00:54:09,698 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:54:09,699 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a4578dcd4830de76f0230a8be7356613 in 42ms, sequenceid=71, compaction requested=true 2023-02-11 00:54:09,699 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a4578dcd4830de76f0230a8be7356613: 2023-02-11 00:54:09,699 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 00:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 00:54:09,701 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,701 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,701 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,701 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 00:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c119c57[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:09,703 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 00:54:09,703 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,705 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,705 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,705 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:09,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,706 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,706 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:09,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,705 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:09,706 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 00:54:09,706 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 00:54:09,706 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:09,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,707 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,707 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,707 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,708 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,708 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 00:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,708 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,708 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 00:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:54:09,711 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 00:54:09,711 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 00:54:09,711 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:09,711 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:09,711 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 00:54:09,711 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:09,713 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,713 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,713 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,713 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,714 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,713 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,714 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:09,714 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,715 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,715 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:09,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,717 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,717 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:09,717 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 00:54:09,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,720 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 00:54:09,720 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:09,720 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:09,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:09,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:09,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:09,720 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:09,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 00:54:09,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 00:54:09,721 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 00:54:09,821 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:09,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:10,084 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 00:54:10,086 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:10,087 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,088 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,089 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,089 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,089 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:10,090 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:10,090 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:10,338 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,339 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:10,339 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:10,339 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:10,341 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-11 00:54:10,342 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:10,342 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:10,342 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:10,342 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:10,343 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:10,346 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 00:54:10,347 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:10,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:10,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:10,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:10,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,350 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:10,350 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,351 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,351 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:10,352 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,352 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:10,352 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,352 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,353 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,353 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-11 00:54:10,354 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:10,354 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:10,356 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076880356 2023-02-11 00:54:10,357 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,357 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:10,361 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:10,362 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:10,362 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:10,362 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076910362 2023-02-11 00:54:10,362 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076970362 2023-02-11 00:54:10,362 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 00:54:10,363 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:10,365 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:10,365 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:10,365 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:10,613 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,613 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:10,613 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:10,613 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:10,613 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:10,613 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:10,613 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:10,613 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:10,613 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:10,614 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:10,616 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 00:54:10,617 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:10,618 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:10,618 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:10,618 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:10,619 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,619 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:10,620 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,620 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,620 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:10,621 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,621 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:10,621 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,621 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,622 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,622 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 00:54:10,622 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:10,623 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:10,623 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076880623 2023-02-11 00:54:10,625 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:10,625 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,628 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:10,628 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:10,629 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:10,629 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076910629 2023-02-11 00:54:10,629 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076970629 2023-02-11 00:54:10,629 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:10,629 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:10,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN}] 2023-02-11 00:54:10,632 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:10,632 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:10,632 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:10,879 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,879 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:10,879 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:10,880 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:10,880 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:10,880 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:10,880 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:10,880 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:10,880 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:10,881 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:10,882 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 00:54:10,882 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:10,883 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:10,883 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:10,883 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:10,884 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,885 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:10,885 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,885 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:10,885 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:10,886 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,886 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:10,886 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,886 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,886 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,886 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 00:54:10,887 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:10,887 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:10,887 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076880887 2023-02-11 00:54:10,889 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:10,889 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:10,891 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:10,892 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:10,892 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:10,892 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076910892 2023-02-11 00:54:10,892 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076970892 2023-02-11 00:54:10,892 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:54:10,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN}] 2023-02-11 00:54:10,892 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:10,894 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:10,895 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=a4578dcd4830de76f0230a8be7356613, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:10,895 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076850895"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076850895"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076850895"}]},"ts":"1676076850895"} 2023-02-11 00:54:10,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:10,897 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:10,897 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:10,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:10,897 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:11,142 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,142 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:11,142 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:11,142 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:11,143 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:11,143 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:11,143 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:11,143 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:11,143 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:11,144 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:11,145 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:11,145 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 00:54:11,146 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:11,146 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:11,146 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:11,147 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:11,147 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,147 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:11,148 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,148 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,148 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:11,149 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,149 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,149 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:11,149 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,150 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,150 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,150 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:11,150 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,150 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:54:11,150 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:11,151 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:11,151 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613 to restore RIT 2023-02-11 00:54:11,151 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076881151 2023-02-11 00:54:11,152 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:11,152 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,154 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:11,155 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:11,155 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:11,155 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076911155 2023-02-11 00:54:11,155 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076971155 2023-02-11 00:54:11,155 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:54:11,155 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=a4578dcd4830de76f0230a8be7356613, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:11,155 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076851155"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076851155"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076851155"}]},"ts":"1676076851155"} 2023-02-11 00:54:11,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:11,157 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:11,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a4578dcd4830de76f0230a8be7356613, disabling compactions & flushes 2023-02-11 00:54:11,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:11,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:11,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. after waiting 0 ms 2023-02-11 00:54:11,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:11,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a4578dcd4830de76f0230a8be7356613 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 00:54:11,323 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/a1219f61b40f4a35b52c34c50e0c568e 2023-02-11 00:54:11,340 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/b928817da7134be89cfb8261f7bba65b 2023-02-11 00:54:11,346 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf1/a1219f61b40f4a35b52c34c50e0c568e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e 2023-02-11 00:54:11,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:54:11,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/.tmp/cf2/b928817da7134be89cfb8261f7bba65b as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b 2023-02-11 00:54:11,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:54:11,358 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a4578dcd4830de76f0230a8be7356613 in 46ms, sequenceid=82, compaction requested=true 2023-02-11 00:54:11,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 00:54:11,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:11,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. 2023-02-11 00:54:11,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a4578dcd4830de76f0230a8be7356613: Waiting for close lock at 1676076851311Running coprocessor pre-close hooks at 1676076851311Disabling compacts and flushes for region at 1676076851311Disabling writes for close at 1676076851311Obtaining lock to block concurrent updates at 1676076851311Preparing flush snapshotting stores in a4578dcd4830de76f0230a8be7356613 at 1676076851311Finished memstore snapshotting testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676076851311Flushing stores of testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. at 1676076851312 (+1 ms)Flushing a4578dcd4830de76f0230a8be7356613/cf1: creating writer at 1676076851313 (+1 ms)Flushing a4578dcd4830de76f0230a8be7356613/cf1: appending metadata at 1676076851316 (+3 ms)Flushing a4578dcd4830de76f0230a8be7356613/cf1: closing flushed file at 1676076851316Flushing a4578dcd4830de76f0230a8be7356613/cf2: creating writer at 1676076851328 (+12 ms)Flushing a4578dcd4830de76f0230a8be7356613/cf2: appending metadata at 1676076851330 (+2 ms)Flushing a4578dcd4830de76f0230a8be7356613/cf2: closing flushed file at 1676076851330Flushing a4578dcd4830de76f0230a8be7356613/cf1: reopening flushed file at 1676076851347 (+17 ms)Flushing a4578dcd4830de76f0230a8be7356613/cf2: reopening flushed file at 1676076851352 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a4578dcd4830de76f0230a8be7356613 in 46ms, sequenceid=82, compaction requested=true at 1676076851358 (+6 ms)Writing region close event to WAL at 1676076851375 (+17 ms)Running coprocessor post-close hooks at 1676076851379 (+4 ms)Closed at 1676076851379 2023-02-11 00:54:11,382 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,382 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=a4578dcd4830de76f0230a8be7356613, regionState=CLOSED 2023-02-11 00:54:11,382 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076851382"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076851382"}]},"ts":"1676076851382"} 2023-02-11 00:54:11,384 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:11,384 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:11,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:11,384 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:11,405 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,405 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:11,405 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:11,405 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:11,406 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:11,406 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:11,406 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:11,406 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:11,406 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:11,407 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:11,408 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:11,408 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 00:54:11,408 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-11 00:54:11,408 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:11,409 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:11,409 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:11,409 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:11,410 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,410 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:11,410 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,410 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,411 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:11,411 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,411 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,411 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:11,412 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:11,412 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,412 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,412 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,412 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:11,412 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,412 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:11,412 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 00:54:11,413 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:11,413 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:11,413 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613 to restore RIT 2023-02-11 00:54:11,413 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076881413 2023-02-11 00:54:11,414 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:11,414 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:11,417 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:11,417 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:11,417 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:11,417 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076911417 2023-02-11 00:54:11,417 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076971417 2023-02-11 00:54:11,417 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:11,418 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=a4578dcd4830de76f0230a8be7356613, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:11,418 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076851418"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076851418"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076851418"}]},"ts":"1676076851418"} 2023-02-11 00:54:11,420 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:11,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-11 00:54:11,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 in 263 msec 2023-02-11 00:54:11,423 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:11,423 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:11,423 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:11,668 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,668 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:11,668 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:11,668 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:11,668 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:11,668 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:11,668 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:11,668 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:11,668 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:11,669 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:11,670 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:11,671 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 00:54:11,671 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:11,671 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:11,672 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:11,672 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:11,672 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:11,673 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,673 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:11,673 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,673 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,674 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:11,674 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,675 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,675 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:11,675 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:11,675 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,675 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,675 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,675 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:11,675 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:11,675 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 00:54:11,676 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:11,676 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:11,676 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613 to restore RIT 2023-02-11 00:54:11,677 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076881677 2023-02-11 00:54:11,677 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:11,678 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,680 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:11,680 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:11,680 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:11,680 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076911680 2023-02-11 00:54:11,680 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076971680 2023-02-11 00:54:11,681 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:11,681 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,682 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:11,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-11 00:54:11,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN in 790 msec 2023-02-11 00:54:11,688 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:11,689 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:11,689 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:11,931 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,931 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:11,931 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:11,931 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:11,931 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:11,931 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:11,931 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:11,931 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:11,932 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:11,932 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:11,934 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:11,934 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:11,934 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:11,934 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:11,934 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:11,935 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:11,935 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:11,935 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,935 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:11,936 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,936 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:11,936 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:11,937 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,937 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:11,937 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:11,937 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:11,937 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:11,937 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:11,937 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:11,938 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:11,938 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:11,938 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076881938 2023-02-11 00:54:11,939 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:11,940 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,942 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:11,942 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:11,942 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:11,943 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076911943 2023-02-11 00:54:11,943 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076971943 2023-02-11 00:54:11,943 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:11,943 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,943 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,943 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:11,948 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:11,959 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=a4578dcd4830de76f0230a8be7356613, threads=14 2023-02-11 00:54:11,960 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,960 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,960 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,960 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,965 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,966 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,966 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,966 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,966 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,966 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,966 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,966 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,967 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:11,976 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,088 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,088 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,112 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,115 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,115 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,129 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,132 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,132 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,134 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,135 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,138 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,140 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,142 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,470 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,470 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region a4578dcd4830de76f0230a8be7356613 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:54:12,509 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:12,509 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:12,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:12,510 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:12,693 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:12,693 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:12,693 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:12,694 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:12,695 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:12,695 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:12,695 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:12,695 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:12,695 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:12,696 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:12,698 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 00:54:12,698 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:12,698 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:12,699 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:12,699 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:12,700 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:12,700 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:12,701 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:12,701 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:12,702 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:12,702 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:12,703 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:12,705 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:12,705 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:12,705 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:12,705 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:12,705 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:12,705 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:12,705 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:12,706 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:12,706 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:12,706 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-11 00:54:12,707 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:12,708 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:12,710 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076882710 2023-02-11 00:54:12,711 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:12,711 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:12,714 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:12,715 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:12,715 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:12,715 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076912715 2023-02-11 00:54:12,715 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076972715 2023-02-11 00:54:12,715 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:12,730 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=a4578dcd4830de76f0230a8be7356613, threads=14 2023-02-11 00:54:12,732 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,732 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,737 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,737 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,738 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,744 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,744 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,745 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,745 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,748 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,752 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,768 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,772 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:12,775 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,027 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,031 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,084 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,088 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,096 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,105 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,106 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,107 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,108 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,110 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,111 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,124 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,126 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3 for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,129 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed for region: a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:13,129 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region a4578dcd4830de76f0230a8be7356613 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:54:13,174 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 00:54:13,174 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:13,182 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:54:13,185 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:54:13,185 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:13,185 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:13,185 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:13,216 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,216 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:13,216 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:13,216 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:13,217 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:13,218 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:13,218 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:13,218 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:13,219 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:13,220 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-11 00:54:13,221 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 00:54:13,221 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:13,222 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:13,249 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:13,250 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:13,253 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:13,253 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:13,255 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:13,256 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:13,256 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:13,256 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:13,257 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:13,258 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,258 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:13,258 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:13,258 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:13,258 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,258 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,258 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,259 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:13,259 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:13,259 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 38 msec 2023-02-11 00:54:13,260 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:13,260 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:13,261 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076883261 2023-02-11 00:54:13,262 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,262 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:13,265 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:13,266 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:13,266 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:13,266 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076913266 2023-02-11 00:54:13,266 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076973266 2023-02-11 00:54:13,266 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:13,270 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:13,272 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:13,272 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:13,272 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:13,516 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:13,517 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:13,517 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:13,517 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:13,518 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:13,518 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:13,518 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:13,518 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:13,518 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:13,519 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:13,520 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 00:54:13,520 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:13,520 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:13,520 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:13,520 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:13,521 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:13,521 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:13,521 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:13,521 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:13,522 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:13,522 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:13,522 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:13,522 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,523 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN 2023-02-11 00:54:13,523 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:13,523 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:13,523 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,523 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,523 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,523 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=a4578dcd4830de76f0230a8be7356613, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:13,523 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure a4578dcd4830de76f0230a8be7356613, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:13,523 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:13,524 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:13,524 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:13,525 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076883525 2023-02-11 00:54:13,526 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:13,526 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,528 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:13,528 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=a4578dcd4830de76f0230a8be7356613, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:13,528 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:13,528 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076913528 2023-02-11 00:54:13,528 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076973528 2023-02-11 00:54:13,529 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:13,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076853530"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676076853530"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676076853530"}]},"ts":"1676076853530"} 2023-02-11 00:54:13,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076853530"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076853530"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076853530"}]},"ts":"1676076853530"} 2023-02-11 00:54:13,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076853530"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076853530"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076853530"}]},"ts":"1676076853530"} 2023-02-11 00:54:13,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=536c8203ff25abc271313ce8e0f84cc7, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=129c0662a2141871c7ffd1b7a430c9e9, ASSIGN}] 2023-02-11 00:54:13,537 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=129c0662a2141871c7ffd1b7a430c9e9, ASSIGN 2023-02-11 00:54:13,538 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=129c0662a2141871c7ffd1b7a430c9e9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:13,539 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=536c8203ff25abc271313ce8e0f84cc7, ASSIGN 2023-02-11 00:54:13,540 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=536c8203ff25abc271313ce8e0f84cc7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:13,688 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 00:54:13,690 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=129c0662a2141871c7ffd1b7a430c9e9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:13,690 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076853690"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076853690"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076853690"}]},"ts":"1676076853690"} 2023-02-11 00:54:13,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 129c0662a2141871c7ffd1b7a430c9e9, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:13,695 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=536c8203ff25abc271313ce8e0f84cc7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:13,695 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076853695"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076853695"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076853695"}]},"ts":"1676076853695"} 2023-02-11 00:54:13,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 536c8203ff25abc271313ce8e0f84cc7, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:13,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:13,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 129c0662a2141871c7ffd1b7a430c9e9, NAME => 'testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 00:54:13,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:13,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:13,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,850 INFO [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,851 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1 2023-02-11 00:54:13,851 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1 2023-02-11 00:54:13,851 INFO [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 129c0662a2141871c7ffd1b7a430c9e9 columnFamilyName cf1 2023-02-11 00:54:13,865 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da-top 2023-02-11 00:54:13,870 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed-top 2023-02-11 00:54:13,874 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed-top 2023-02-11 00:54:13,879 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e-top 2023-02-11 00:54:13,883 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef-top 2023-02-11 00:54:13,887 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e-top 2023-02-11 00:54:13,891 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3-top 2023-02-11 00:54:13,891 INFO [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(310): Store=129c0662a2141871c7ffd1b7a430c9e9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:13,892 INFO [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,892 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2 2023-02-11 00:54:13,892 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2 2023-02-11 00:54:13,893 INFO [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 129c0662a2141871c7ffd1b7a430c9e9 columnFamilyName cf2 2023-02-11 00:54:13,907 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0-top 2023-02-11 00:54:13,911 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e-top 2023-02-11 00:54:13,915 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f-top 2023-02-11 00:54:13,919 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255-top 2023-02-11 00:54:13,923 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b-top 2023-02-11 00:54:13,928 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457-top 2023-02-11 00:54:13,933 DEBUG [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd-top 2023-02-11 00:54:13,933 INFO [StoreOpener-129c0662a2141871c7ffd1b7a430c9e9-1] regionserver.HStore(310): Store=129c0662a2141871c7ffd1b7a430c9e9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:13,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:13,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:13,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 129c0662a2141871c7ffd1b7a430c9e9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61387218, jitterRate=-0.08525916934013367}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:13,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 129c0662a2141871c7ffd1b7a430c9e9: Running coprocessor pre-open hook at 1676076853848Writing region info on filesystem at 1676076853848Initializing all the Stores at 1676076853849 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076853849Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076853849Cleaning up temporary data from old regions at 1676076853936 (+87 ms)Cleaning up detritus from prior splits at 1676076853936Running coprocessor post-open hooks at 1676076853939 (+3 ms)Region opened successfully at 1676076853939 2023-02-11 00:54:13,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9., pid=33, masterSystemTime=1676076853844 2023-02-11 00:54:13,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:13,941 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:13,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:54:13,941 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:13,941 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:13,941 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1912): 129c0662a2141871c7ffd1b7a430c9e9/cf1 is initiating minor compaction (all files) 2023-02-11 00:54:13,941 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:13,941 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1912): 129c0662a2141871c7ffd1b7a430c9e9/cf2 is initiating minor compaction (all files) 2023-02-11 00:54:13,941 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 129c0662a2141871c7ffd1b7a430c9e9/cf1 in testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:13,941 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 129c0662a2141871c7ffd1b7a430c9e9/cf2 in testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:13,941 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e-top] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/.tmp, totalSize=34.9 K 2023-02-11 00:54:13,941 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b-top] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/.tmp, totalSize=34.9 K 2023-02-11 00:54:13,942 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076848625 2023-02-11 00:54:13,942 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076848625 2023-02-11 00:54:13,942 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076848819 2023-02-11 00:54:13,943 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076848819 2023-02-11 00:54:13,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:13,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:13,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:13,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 536c8203ff25abc271313ce8e0f84cc7, NAME => 'testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 00:54:13,943 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076849029 2023-02-11 00:54:13,943 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076849029 2023-02-11 00:54:13,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:13,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:13,944 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=129c0662a2141871c7ffd1b7a430c9e9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:13,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:13,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:13,944 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:13,944 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076853944"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076853944"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076853944"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076853944"}]},"ts":"1676076853944"} 2023-02-11 00:54:13,944 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676076849229 2023-02-11 00:54:13,944 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676076849229 2023-02-11 00:54:13,945 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676076849430 2023-02-11 00:54:13,945 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676076849430 2023-02-11 00:54:13,945 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676076849635 2023-02-11 00:54:13,946 INFO [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:13,946 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676076849635 2023-02-11 00:54:13,946 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076849823 2023-02-11 00:54:13,947 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076849823 2023-02-11 00:54:13,948 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1 2023-02-11 00:54:13,949 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1 2023-02-11 00:54:13,949 INFO [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 536c8203ff25abc271313ce8e0f84cc7 columnFamilyName cf1 2023-02-11 00:54:13,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-11 00:54:13,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 129c0662a2141871c7ffd1b7a430c9e9, server=jenkins-hbase4.apache.org,38317,1676076840434 in 254 msec 2023-02-11 00:54:13,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=129c0662a2141871c7ffd1b7a430c9e9, ASSIGN in 415 msec 2023-02-11 00:54:13,965 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] throttle.PressureAwareThroughputController(145): 129c0662a2141871c7ffd1b7a430c9e9#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:13,966 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] throttle.PressureAwareThroughputController(145): 129c0662a2141871c7ffd1b7a430c9e9#cf2#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:13,969 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da-bottom 2023-02-11 00:54:13,984 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed-bottom 2023-02-11 00:54:13,985 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed-bottom 2023-02-11 00:54:13,993 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e-bottom 2023-02-11 00:54:13,998 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef-bottom 2023-02-11 00:54:14,000 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/.tmp/cf1/836dad4aac8b4457959597e7b357b594 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/836dad4aac8b4457959597e7b357b594 2023-02-11 00:54:14,002 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/.tmp/cf2/214537a1e3ae4cec8157a063455c06c3 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/214537a1e3ae4cec8157a063455c06c3 2023-02-11 00:54:14,005 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e-bottom 2023-02-11 00:54:14,008 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 129c0662a2141871c7ffd1b7a430c9e9/cf1 of 129c0662a2141871c7ffd1b7a430c9e9 into 836dad4aac8b4457959597e7b357b594(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:14,008 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 129c0662a2141871c7ffd1b7a430c9e9: 2023-02-11 00:54:14,008 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9., storeName=129c0662a2141871c7ffd1b7a430c9e9/cf1, priority=9, startTime=1676076853940; duration=0sec 2023-02-11 00:54:14,008 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 129c0662a2141871c7ffd1b7a430c9e9/cf2 of 129c0662a2141871c7ffd1b7a430c9e9 into 214537a1e3ae4cec8157a063455c06c3(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:14,008 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:14,008 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 129c0662a2141871c7ffd1b7a430c9e9: 2023-02-11 00:54:14,008 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9., storeName=129c0662a2141871c7ffd1b7a430c9e9/cf2, priority=9, startTime=1676076853941; duration=0sec 2023-02-11 00:54:14,009 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:14,012 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3-bottom 2023-02-11 00:54:14,012 INFO [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(310): Store=536c8203ff25abc271313ce8e0f84cc7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:14,013 INFO [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:14,013 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2 2023-02-11 00:54:14,013 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2 2023-02-11 00:54:14,014 INFO [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 536c8203ff25abc271313ce8e0f84cc7 columnFamilyName cf2 2023-02-11 00:54:14,028 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0-bottom 2023-02-11 00:54:14,031 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e-bottom 2023-02-11 00:54:14,035 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f-bottom 2023-02-11 00:54:14,039 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255-bottom 2023-02-11 00:54:14,043 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b-bottom 2023-02-11 00:54:14,046 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457-bottom 2023-02-11 00:54:14,050 DEBUG [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd-bottom 2023-02-11 00:54:14,050 INFO [StoreOpener-536c8203ff25abc271313ce8e0f84cc7-1] regionserver.HStore(310): Store=536c8203ff25abc271313ce8e0f84cc7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:14,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:14,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:14,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:14,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:14,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 536c8203ff25abc271313ce8e0f84cc7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73562797, jitterRate=0.09617109596729279}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:14,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 536c8203ff25abc271313ce8e0f84cc7: Running coprocessor pre-open hook at 1676076853944Writing region info on filesystem at 1676076853944Initializing all the Stores at 1676076853945 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076853945Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076853946 (+1 ms)Cleaning up temporary data from old regions at 1676076854052 (+106 ms)Cleaning up detritus from prior splits at 1676076854053 (+1 ms)Running coprocessor post-open hooks at 1676076854055 (+2 ms)Region opened successfully at 1676076854055 2023-02-11 00:54:14,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7., pid=34, masterSystemTime=1676076853844 2023-02-11 00:54:14,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:14,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:54:14,057 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:14,057 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:14,057 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,057 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1912): 536c8203ff25abc271313ce8e0f84cc7/cf1 is initiating minor compaction (all files) 2023-02-11 00:54:14,057 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,057 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1912): 536c8203ff25abc271313ce8e0f84cc7/cf2 is initiating minor compaction (all files) 2023-02-11 00:54:14,057 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 536c8203ff25abc271313ce8e0f84cc7/cf1 in testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,057 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 536c8203ff25abc271313ce8e0f84cc7/cf2 in testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,058 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/.tmp, totalSize=34.9 K 2023-02-11 00:54:14,058 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/.tmp, totalSize=34.9 K 2023-02-11 00:54:14,058 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076848625 2023-02-11 00:54:14,058 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076848625 2023-02-11 00:54:14,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,059 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076848819 2023-02-11 00:54:14,059 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076848819 2023-02-11 00:54:14,059 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076849029 2023-02-11 00:54:14,059 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=536c8203ff25abc271313ce8e0f84cc7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:14,059 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076849029 2023-02-11 00:54:14,060 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076854059"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076854059"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076854059"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076854059"}]},"ts":"1676076854059"} 2023-02-11 00:54:14,060 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076849229 2023-02-11 00:54:14,060 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076849229 2023-02-11 00:54:14,060 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076849430 2023-02-11 00:54:14,061 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076849430 2023-02-11 00:54:14,061 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076849635 2023-02-11 00:54:14,061 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076849635 2023-02-11 00:54:14,061 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076849823 2023-02-11 00:54:14,062 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076849823 2023-02-11 00:54:14,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-11 00:54:14,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 536c8203ff25abc271313ce8e0f84cc7, server=jenkins-hbase4.apache.org,38317,1676076840434 in 366 msec 2023-02-11 00:54:14,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-11 00:54:14,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=536c8203ff25abc271313ce8e0f84cc7, ASSIGN in 531 msec 2023-02-11 00:54:14,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 in 3.9830 sec 2023-02-11 00:54:14,077 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] throttle.PressureAwareThroughputController(145): 536c8203ff25abc271313ce8e0f84cc7#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:14,077 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] throttle.PressureAwareThroughputController(145): 536c8203ff25abc271313ce8e0f84cc7#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:14,099 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/.tmp/cf1/0e4d3296b0f740fd987c757f1a2f2721 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/0e4d3296b0f740fd987c757f1a2f2721 2023-02-11 00:54:14,100 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/.tmp/cf2/31d944fd4d7949e899bc7a39abf94b39 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/31d944fd4d7949e899bc7a39abf94b39 2023-02-11 00:54:14,107 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 536c8203ff25abc271313ce8e0f84cc7/cf1 of 536c8203ff25abc271313ce8e0f84cc7 into 0e4d3296b0f740fd987c757f1a2f2721(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:14,107 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 536c8203ff25abc271313ce8e0f84cc7: 2023-02-11 00:54:14,107 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7., storeName=536c8203ff25abc271313ce8e0f84cc7/cf1, priority=9, startTime=1676076854056; duration=0sec 2023-02-11 00:54:14,107 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:14,111 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 536c8203ff25abc271313ce8e0f84cc7/cf2 of 536c8203ff25abc271313ce8e0f84cc7 into 31d944fd4d7949e899bc7a39abf94b39(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:14,111 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 536c8203ff25abc271313ce8e0f84cc7: 2023-02-11 00:54:14,111 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7., storeName=536c8203ff25abc271313ce8e0f84cc7/cf2, priority=9, startTime=1676076854057; duration=0sec 2023-02-11 00:54:14,111 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:14,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38317] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:45472 deadline: 1676076914280, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613. is not online on jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:14,392 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 00:54:14,401 INFO [Listener at localhost/35701] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 00:54:14,402 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 00:54:14,422 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:14,424 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-11 00:54:14,424 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-11 00:54:14,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:14,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 00:54:14,428 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076854428"}]},"ts":"1676076854428"} 2023-02-11 00:54:14,430 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-11 00:54:14,431 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-11 00:54:14,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=536c8203ff25abc271313ce8e0f84cc7, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=129c0662a2141871c7ffd1b7a430c9e9, UNASSIGN}] 2023-02-11 00:54:14,433 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=129c0662a2141871c7ffd1b7a430c9e9, UNASSIGN 2023-02-11 00:54:14,434 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=129c0662a2141871c7ffd1b7a430c9e9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:14,434 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076854434"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076854434"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076854434"}]},"ts":"1676076854434"} 2023-02-11 00:54:14,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 129c0662a2141871c7ffd1b7a430c9e9, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:14,438 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=536c8203ff25abc271313ce8e0f84cc7, UNASSIGN 2023-02-11 00:54:14,439 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=536c8203ff25abc271313ce8e0f84cc7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:14,439 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076854439"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076854439"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076854439"}]},"ts":"1676076854439"} 2023-02-11 00:54:14,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 536c8203ff25abc271313ce8e0f84cc7, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:14,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 00:54:14,587 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:14,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 536c8203ff25abc271313ce8e0f84cc7, disabling compactions & flushes 2023-02-11 00:54:14,588 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. after waiting 0 ms 2023-02-11 00:54:14,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,599 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e-bottom] to archive 2023-02-11 00:54:14,600 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:14,602 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,603 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,604 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,606 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,607 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,608 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,609 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,611 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b-bottom] to archive 2023-02-11 00:54:14,612 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:14,614 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,615 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,616 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,617 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,618 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,619 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,620 DEBUG [StoreCloser-testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,624 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:54:14,624 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:14,625 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7. 2023-02-11 00:54:14,625 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 536c8203ff25abc271313ce8e0f84cc7: Waiting for close lock at 1676076854587Running coprocessor pre-close hooks at 1676076854587Disabling compacts and flushes for region at 1676076854587Disabling writes for close at 1676076854588 (+1 ms)Writing region close event to WAL at 1676076854621 (+33 ms)Running coprocessor post-close hooks at 1676076854624 (+3 ms)Closed at 1676076854625 (+1 ms) 2023-02-11 00:54:14,626 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:14,627 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:14,627 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 129c0662a2141871c7ffd1b7a430c9e9, disabling compactions & flushes 2023-02-11 00:54:14,627 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:14,627 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:14,627 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. after waiting 0 ms 2023-02-11 00:54:14,627 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:14,627 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=536c8203ff25abc271313ce8e0f84cc7, regionState=CLOSED 2023-02-11 00:54:14,627 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076854627"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076854627"}]},"ts":"1676076854627"} 2023-02-11 00:54:14,628 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e-top] to archive 2023-02-11 00:54:14,629 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:14,631 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-11 00:54:14,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 536c8203ff25abc271313ce8e0f84cc7, server=jenkins-hbase4.apache.org,38317,1676076840434 in 189 msec 2023-02-11 00:54:14,632 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=536c8203ff25abc271313ce8e0f84cc7, UNASSIGN in 200 msec 2023-02-11 00:54:14,633 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,634 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,636 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,637 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,638 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,640 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b-top] to archive 2023-02-11 00:54:14,640 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:14,642 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,643 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,644 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,645 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,646 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,647 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,648 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,651 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:54:14,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:14,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9. 2023-02-11 00:54:14,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 129c0662a2141871c7ffd1b7a430c9e9: Waiting for close lock at 1676076854627Running coprocessor pre-close hooks at 1676076854627Disabling compacts and flushes for region at 1676076854627Disabling writes for close at 1676076854627Writing region close event to WAL at 1676076854649 (+22 ms)Running coprocessor post-close hooks at 1676076854652 (+3 ms)Closed at 1676076854652 2023-02-11 00:54:14,654 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:14,654 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=129c0662a2141871c7ffd1b7a430c9e9, regionState=CLOSED 2023-02-11 00:54:14,654 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076854654"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076854654"}]},"ts":"1676076854654"} 2023-02-11 00:54:14,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-11 00:54:14,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 129c0662a2141871c7ffd1b7a430c9e9, server=jenkins-hbase4.apache.org,38317,1676076840434 in 221 msec 2023-02-11 00:54:14,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-11 00:54:14,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=129c0662a2141871c7ffd1b7a430c9e9, UNASSIGN in 225 msec 2023-02-11 00:54:14,660 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076854660"}]},"ts":"1676076854660"} 2023-02-11 00:54:14,662 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-11 00:54:14,663 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-11 00:54:14,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 240 msec 2023-02-11 00:54:14,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 00:54:14,731 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-11 00:54:14,732 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-11 00:54:14,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:14,734 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:14,735 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:14,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-11 00:54:14,741 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,741 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:14,741 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:14,743 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/recovered.edits] 2023-02-11 00:54:14,744 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/recovered.edits] 2023-02-11 00:54:14,744 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/recovered.edits] 2023-02-11 00:54:14,748 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/.regioninfo to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/.regioninfo 2023-02-11 00:54:14,748 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/.regioninfo to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/.regioninfo 2023-02-11 00:54:14,749 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3155732a33c14a41a0a436742c7bc1da 2023-02-11 00:54:14,750 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/3d9972b749ae4f1d840591ba0515b3ed 2023-02-11 00:54:14,751 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,752 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/3155732a33c14a41a0a436742c7bc1da.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,752 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/7471952cfabc4abea2f5c2fa96ce09ed 2023-02-11 00:54:14,753 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,753 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/3d9972b749ae4f1d840591ba0515b3ed.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,753 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/a1219f61b40f4a35b52c34c50e0c568e 2023-02-11 00:54:14,755 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,755 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/7471952cfabc4abea2f5c2fa96ce09ed.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,755 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/ba378566ea874463b18342836eb906ef 2023-02-11 00:54:14,756 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,756 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/a1219f61b40f4a35b52c34c50e0c568e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,757 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/c68fa8a7cb3541faa8fa033756bb1a2e 2023-02-11 00:54:14,758 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,758 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/ba378566ea874463b18342836eb906ef.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,758 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf1/cb7632271f904b0680638b360a4334f3 2023-02-11 00:54:14,759 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,759 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/c68fa8a7cb3541faa8fa033756bb1a2e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,761 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,761 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf1/cb7632271f904b0680638b360a4334f3.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,762 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/1551b7d98f8b44f5b3df7284af3b53b0 2023-02-11 00:54:14,763 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e 2023-02-11 00:54:14,765 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,766 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/88c00ba48f834f33bcd361eacc97762f 2023-02-11 00:54:14,766 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/1551b7d98f8b44f5b3df7284af3b53b0.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,767 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,768 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/90466a24df1d4ca2b8b7a8b112578255 2023-02-11 00:54:14,768 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/3766fd3ee70b4cfc9b8bfe12877fbb5e.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,769 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,769 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/b928817da7134be89cfb8261f7bba65b 2023-02-11 00:54:14,770 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/88c00ba48f834f33bcd361eacc97762f.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,771 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,772 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/e4cb3e45f9d843d7a7edfcc5cce85457 2023-02-11 00:54:14,772 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/90466a24df1d4ca2b8b7a8b112578255.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,774 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,774 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/cf2/fb2d53d58cbe489ca60e05194f85f1dd 2023-02-11 00:54:14,774 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/b928817da7134be89cfb8261f7bba65b.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,776 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,776 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/e4cb3e45f9d843d7a7edfcc5cce85457.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,777 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/129c0662a2141871c7ffd1b7a430c9e9/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,778 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/536c8203ff25abc271313ce8e0f84cc7/cf2/fb2d53d58cbe489ca60e05194f85f1dd.a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,778 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/recovered.edits/85.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613/recovered.edits/85.seqid 2023-02-11 00:54:14,779 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/a4578dcd4830de76f0230a8be7356613 2023-02-11 00:54:14,781 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/836dad4aac8b4457959597e7b357b594 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf1/836dad4aac8b4457959597e7b357b594 2023-02-11 00:54:14,781 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/0e4d3296b0f740fd987c757f1a2f2721 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf1/0e4d3296b0f740fd987c757f1a2f2721 2023-02-11 00:54:14,784 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/214537a1e3ae4cec8157a063455c06c3 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/cf2/214537a1e3ae4cec8157a063455c06c3 2023-02-11 00:54:14,784 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/31d944fd4d7949e899bc7a39abf94b39 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/cf2/31d944fd4d7949e899bc7a39abf94b39 2023-02-11 00:54:14,787 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9/recovered.edits/90.seqid 2023-02-11 00:54:14,788 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7/recovered.edits/90.seqid 2023-02-11 00:54:14,788 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:14,788 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitWithoutPONR/536c8203ff25abc271313ce8e0f84cc7 2023-02-11 00:54:14,788 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-11 00:54:14,791 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:14,793 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-11 00:54:14,795 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-11 00:54:14,796 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:14,796 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-11 00:54:14,796 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076854796"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:14,796 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076854796"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:14,796 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076854796"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:14,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 00:54:14,798 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a4578dcd4830de76f0230a8be7356613, NAME => 'testSplitWithoutPONR,,1676076848120.a4578dcd4830de76f0230a8be7356613.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 536c8203ff25abc271313ce8e0f84cc7, NAME => 'testSplitWithoutPONR,,1676076850086.536c8203ff25abc271313ce8e0f84cc7.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 129c0662a2141871c7ffd1b7a430c9e9, NAME => 'testSplitWithoutPONR,41,1676076850086.129c0662a2141871c7ffd1b7a430c9e9.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 00:54:14,799 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-11 00:54:14,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076854799"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:14,801 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-11 00:54:14,803 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:14,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 71 msec 2023-02-11 00:54:14,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-11 00:54:14,837 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-11 00:54:14,852 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=399 (was 395) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:48060 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:49918 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:33114 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:50246 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:49906 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:50308 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:40954 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79353ebb-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41704 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:40950 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47824 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x79353ebb-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47716 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47856 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41416 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41692 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:41720 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_399292632_15 at /127.0.0.1:47806 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=858 (was 835) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=209 (was 227), ProcessCount=168 (was 168), AvailableMemoryMB=3278 (was 3413) 2023-02-11 00:54:14,866 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=399, OpenFileDescriptor=858, MaxFileDescriptor=60000, SystemLoadAverage=209, ProcessCount=168, AvailableMemoryMB=3277 2023-02-11 00:54:14,867 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:14,875 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:54:14,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:54:14,877 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:14,878 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:54:14,879 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:54:14,881 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:14,881 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 empty. 2023-02-11 00:54:14,882 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:14,882 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-11 00:54:14,893 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:14,894 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 517bceec69b190e0374f76671f87d5d1, NAME => 'testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:54:14,904 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:14,904 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 517bceec69b190e0374f76671f87d5d1, disabling compactions & flushes 2023-02-11 00:54:14,904 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:14,904 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:14,904 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. after waiting 0 ms 2023-02-11 00:54:14,904 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:14,904 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:14,904 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 517bceec69b190e0374f76671f87d5d1: Waiting for close lock at 1676076854904Disabling compacts and flushes for region at 1676076854904Disabling writes for close at 1676076854904Writing region close event to WAL at 1676076854904Closed at 1676076854904 2023-02-11 00:54:14,906 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:54:14,907 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076854907"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076854907"}]},"ts":"1676076854907"} 2023-02-11 00:54:14,909 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:54:14,910 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:54:14,910 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076854910"}]},"ts":"1676076854910"} 2023-02-11 00:54:14,911 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-11 00:54:14,914 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:14,915 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:14,915 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:14,915 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:14,915 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:54:14,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN}] 2023-02-11 00:54:14,918 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:14,919 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33641,1676076840386; forceNewPlan=false, retain=false 2023-02-11 00:54:15,070 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:15,071 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,071 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076855071"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076855071"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076855071"}]},"ts":"1676076855071"} 2023-02-11 00:54:15,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:15,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 517bceec69b190e0374f76671f87d5d1, NAME => 'testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:15,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:15,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:15,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:15,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:15,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:15,235 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:15,236 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1 2023-02-11 00:54:15,236 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1 2023-02-11 00:54:15,236 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 517bceec69b190e0374f76671f87d5d1 columnFamilyName cf1 2023-02-11 00:54:15,237 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(310): Store=517bceec69b190e0374f76671f87d5d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:15,237 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:15,238 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2 2023-02-11 00:54:15,238 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2 2023-02-11 00:54:15,239 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 517bceec69b190e0374f76671f87d5d1 columnFamilyName cf2 2023-02-11 00:54:15,239 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(310): Store=517bceec69b190e0374f76671f87d5d1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:15,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:15,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:15,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:15,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:15,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:15,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 517bceec69b190e0374f76671f87d5d1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59364139, jitterRate=-0.11540539562702179}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:15,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 517bceec69b190e0374f76671f87d5d1: Running coprocessor pre-open hook at 1676076855234Writing region info on filesystem at 1676076855234Initializing all the Stores at 1676076855234Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076855234Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076855235 (+1 ms)Cleaning up temporary data from old regions at 1676076855241 (+6 ms)Cleaning up detritus from prior splits at 1676076855242 (+1 ms)Running coprocessor post-open hooks at 1676076855246 (+4 ms)Region opened successfully at 1676076855246 2023-02-11 00:54:15,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1., pid=43, masterSystemTime=1676076855225 2023-02-11 00:54:15,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,249 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,249 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076855249"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076855249"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076855249"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076855249"}]},"ts":"1676076855249"} 2023-02-11 00:54:15,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-11 00:54:15,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 in 177 msec 2023-02-11 00:54:15,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-11 00:54:15,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN in 337 msec 2023-02-11 00:54:15,255 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:54:15,255 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076855255"}]},"ts":"1676076855255"} 2023-02-11 00:54:15,256 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-11 00:54:15,262 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:54:15,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 387 msec 2023-02-11 00:54:15,386 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:15,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 00:54:15,388 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 00:54:15,388 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,388 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 00:54:15,388 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:54:15,388 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,388 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,389 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,389 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,389 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,389 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,390 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,390 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:15,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,396 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,396 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,396 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,396 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,396 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:15,397 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,397 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:15,397 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. started... 2023-02-11 00:54:15,398 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 517bceec69b190e0374f76671f87d5d1 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 00:54:15,399 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,401 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,401 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,401 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,401 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,401 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,411 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/c44cbfe5d17a400abb72cb0836c08d55 2023-02-11 00:54:15,425 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/d2282319e6514486adb4b895349ff1f0 2023-02-11 00:54:15,430 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/c44cbfe5d17a400abb72cb0836c08d55 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55 2023-02-11 00:54:15,434 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:54:15,434 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/d2282319e6514486adb4b895349ff1f0 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0 2023-02-11 00:54:15,438 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:54:15,439 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 517bceec69b190e0374f76671f87d5d1 in 41ms, sequenceid=6, compaction requested=false 2023-02-11 00:54:15,439 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-11 00:54:15,439 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 517bceec69b190e0374f76671f87d5d1: 2023-02-11 00:54:15,439 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,439 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:15,440 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:15,440 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,440 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,440 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,441 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,441 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:15,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:15,442 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,442 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:15,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:15,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:15,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 00:54:15,444 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:54:15,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ede0767[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:15,444 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,445 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:15,445 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,445 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:15,447 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,447 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,447 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,447 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:15,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:15,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:15,450 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,450 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,450 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:15,451 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,451 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,451 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:15,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:54:15,455 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 00:54:15,455 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:15,455 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:15,455 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 00:54:15,455 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:15,457 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,457 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,457 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,457 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,457 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,460 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,460 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:15,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,460 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,461 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:15,461 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:15,463 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,463 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,463 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,467 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,467 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 00:54:15,467 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,467 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,468 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,568 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:15,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:15,586 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:15,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:15,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 00:54:15,588 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 00:54:15,588 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,588 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 00:54:15,588 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:54:15,589 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,589 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,591 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,591 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,591 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,591 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,591 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,592 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:15,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,593 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,593 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,593 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,593 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,593 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,594 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,594 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,594 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:15,594 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,594 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,594 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. started... 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,594 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,595 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,595 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 517bceec69b190e0374f76671f87d5d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:15,595 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,596 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,596 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,596 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,597 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,597 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,605 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/098ac1c41a8d40e0b1193154f0a4eab0 2023-02-11 00:54:15,616 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/3bc9b70646ae422a9394a219a645edc7 2023-02-11 00:54:15,621 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/098ac1c41a8d40e0b1193154f0a4eab0 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0 2023-02-11 00:54:15,625 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:54:15,626 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/3bc9b70646ae422a9394a219a645edc7 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7 2023-02-11 00:54:15,630 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:54:15,631 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 517bceec69b190e0374f76671f87d5d1 in 36ms, sequenceid=19, compaction requested=false 2023-02-11 00:54:15,631 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 517bceec69b190e0374f76671f87d5d1: 2023-02-11 00:54:15,631 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,631 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:15,631 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:15,631 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,631 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,631 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,633 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,633 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:15,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:15,633 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,633 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:15,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:15,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:15,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 00:54:15,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e1f0a5f[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:15,635 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:54:15,635 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,637 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,637 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,637 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,638 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,638 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:15,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:15,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,638 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,642 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,642 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:15,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:15,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:15,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:15,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:15,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:15,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:54:15,646 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 00:54:15,646 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:15,646 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:15,646 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 00:54:15,646 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:15,647 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,647 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:15,648 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,648 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,647 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:15,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,648 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,649 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,650 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,651 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:15,651 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,653 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,653 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,653 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,654 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,654 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,654 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 00:54:15,654 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:15,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,654 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,654 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,653 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,654 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:15,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,653 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,654 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:15,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,654 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,654 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,655 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,754 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:15,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:15,770 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:15,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:15,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 00:54:15,772 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 00:54:15,772 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,772 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 00:54:15,772 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:54:15,773 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,773 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,774 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,775 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,775 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,775 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,775 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,775 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:15,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,775 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,776 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,776 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,777 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,777 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,777 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. started... 2023-02-11 00:54:15,777 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 517bceec69b190e0374f76671f87d5d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:15,778 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,779 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,779 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,779 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,779 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,779 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,788 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/9db8268ca1ce4147b943228156b6269b 2023-02-11 00:54:15,805 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/4609fa2424e5493a82109ee1e9e7d7b0 2023-02-11 00:54:15,810 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/9db8268ca1ce4147b943228156b6269b as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b 2023-02-11 00:54:15,814 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:54:15,815 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/4609fa2424e5493a82109ee1e9e7d7b0 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0 2023-02-11 00:54:15,818 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:54:15,819 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 517bceec69b190e0374f76671f87d5d1 in 42ms, sequenceid=32, compaction requested=true 2023-02-11 00:54:15,819 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 517bceec69b190e0374f76671f87d5d1: 2023-02-11 00:54:15,819 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,819 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:15,820 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:15,820 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,820 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,820 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,821 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,821 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:15,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:15,822 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,822 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:15,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:15,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:15,823 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 00:54:15,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7652077[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:15,824 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:54:15,824 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,826 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,826 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,826 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,826 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:15,826 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:15,826 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:15,827 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:15,827 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:15,828 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,828 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,828 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,828 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,828 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,828 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:15,828 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,829 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,829 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,829 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:15,829 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:54:15,829 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:15,829 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 00:54:15,829 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:15,829 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:15,829 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 00:54:15,829 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:15,830 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,830 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:15,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:15,831 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,830 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,830 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,831 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,831 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:15,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,831 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,831 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,832 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:15,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:15,834 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:15,835 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,835 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,846 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 00:54:15,846 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:15,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,846 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:15,846 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,846 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:15,846 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,846 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,846 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:15,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,846 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:15,846 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,847 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,847 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:15,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:15,847 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:15,847 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:15,847 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,847 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,947 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:15,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:15,963 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:15,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:15,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 00:54:15,964 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 00:54:15,964 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,965 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 00:54:15,965 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:54:15,965 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,965 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,966 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,966 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,966 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,966 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:15,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:15,967 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:15,967 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:15,968 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:15,968 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,968 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:15,968 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:15,969 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:15,969 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:15,969 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:15,970 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,970 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,970 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,970 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,970 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:15,971 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:15,973 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:15,973 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:15,973 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:15,973 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. started... 2023-02-11 00:54:15,973 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 517bceec69b190e0374f76671f87d5d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:15,983 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/d1448290ffc64f938c9a94a5c20cba5f 2023-02-11 00:54:15,996 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/07ce53397b494ba49b5976a67c13795c 2023-02-11 00:54:16,001 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/d1448290ffc64f938c9a94a5c20cba5f as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f 2023-02-11 00:54:16,005 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:54:16,006 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/07ce53397b494ba49b5976a67c13795c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c 2023-02-11 00:54:16,011 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:54:16,016 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 517bceec69b190e0374f76671f87d5d1 in 43ms, sequenceid=45, compaction requested=true 2023-02-11 00:54:16,016 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 517bceec69b190e0374f76671f87d5d1: 2023-02-11 00:54:16,016 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:16,016 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:16,016 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:16,017 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,017 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:16,017 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:16,020 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,020 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,020 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,020 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:16,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 00:54:16,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7766f7e8[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:16,022 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:54:16,022 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,024 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:16,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,024 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:16,026 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,026 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,026 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,026 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,026 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,027 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,027 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,027 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,027 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,027 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:16,032 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:54:16,032 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 00:54:16,032 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:16,032 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:16,032 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 00:54:16,032 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:16,035 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,035 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,035 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,035 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,035 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,036 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,035 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,036 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,036 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,037 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,038 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,039 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,039 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,041 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,041 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,041 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,041 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,041 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,041 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 00:54:16,041 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:16,041 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,041 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,041 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,041 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,042 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:16,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,042 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:16,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,042 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,042 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,042 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,042 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,042 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,142 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:16,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:16,159 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:16,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:16,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 00:54:16,160 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 00:54:16,160 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:16,160 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 00:54:16,161 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:54:16,161 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,161 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,162 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,162 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,162 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,163 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,163 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:16,162 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:16,164 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:16,164 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:16,165 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:16,165 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:16,164 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:16,165 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:16,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:16,165 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:16,165 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:16,165 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:16,165 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:16,165 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:16,165 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:16,165 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. started... 2023-02-11 00:54:16,165 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 517bceec69b190e0374f76671f87d5d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:16,166 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,168 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,168 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,168 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:16,168 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,168 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:16,176 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/4e27c584057b4b4c88727a838f12d532 2023-02-11 00:54:16,188 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/634d93347ec74d2c837f25e586834df1 2023-02-11 00:54:16,192 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/4e27c584057b4b4c88727a838f12d532 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532 2023-02-11 00:54:16,196 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:54:16,197 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/634d93347ec74d2c837f25e586834df1 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1 2023-02-11 00:54:16,200 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:54:16,201 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 517bceec69b190e0374f76671f87d5d1 in 36ms, sequenceid=58, compaction requested=true 2023-02-11 00:54:16,201 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 517bceec69b190e0374f76671f87d5d1: 2023-02-11 00:54:16,201 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:16,202 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:16,202 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:16,202 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,202 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:16,202 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:16,203 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,203 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,203 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,204 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 00:54:16,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@691d335c[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:16,205 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:54:16,205 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,206 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:16,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,206 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,206 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:16,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,206 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,207 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:16,207 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,207 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,207 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:16,208 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,208 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,208 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,212 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,212 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,212 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,212 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,212 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,212 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,212 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:16,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:54:16,215 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 00:54:16,215 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:16,215 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:16,215 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 00:54:16,215 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:16,217 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,217 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,217 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,217 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,218 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,218 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,220 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,220 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,220 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,222 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,223 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 00:54:16,223 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:16,223 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,223 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,224 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,224 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,224 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,224 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,224 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,224 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,323 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:16,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:16,338 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:16,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:16,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 00:54:16,340 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 00:54:16,340 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:16,340 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 00:54:16,340 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:54:16,340 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,340 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,342 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,342 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,342 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,342 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,342 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,342 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:16,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:16,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:16,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:16,343 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:16,344 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:16,344 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:16,344 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. started... 2023-02-11 00:54:16,344 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 517bceec69b190e0374f76671f87d5d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:16,346 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,347 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,347 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,347 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:16,347 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,347 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:16,353 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/10d8c709e2234bce84e59f274912fd64 2023-02-11 00:54:16,364 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/b835a27056274355bd1dccc95d5fa285 2023-02-11 00:54:16,369 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/10d8c709e2234bce84e59f274912fd64 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64 2023-02-11 00:54:16,373 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:54:16,373 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/b835a27056274355bd1dccc95d5fa285 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285 2023-02-11 00:54:16,377 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:54:16,378 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 517bceec69b190e0374f76671f87d5d1 in 34ms, sequenceid=71, compaction requested=true 2023-02-11 00:54:16,378 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 517bceec69b190e0374f76671f87d5d1: 2023-02-11 00:54:16,378 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:16,378 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:16,378 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:16,378 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,378 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 00:54:16,378 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 00:54:16,379 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,380 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,380 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,380 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 00:54:16,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fb681d2[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:16,382 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:54:16,382 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,383 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,383 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,383 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,383 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:16,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,383 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,384 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:16,384 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:54:16,384 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 00:54:16,384 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:16,386 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,386 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,386 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,386 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 00:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,387 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 00:54:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:16,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:54:16,390 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 00:54:16,390 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:16,390 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:16,390 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 00:54:16,390 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:16,391 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,391 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,391 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,391 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:16,392 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,392 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,392 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,393 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,393 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,394 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:16,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:16,396 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 00:54:16,396 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,397 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,406 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,406 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 00:54:16,406 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:16,406 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 00:54:16,407 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 00:54:16,507 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:16,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:16,768 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 00:54:16,769 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:16,770 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:16,771 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:16,771 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:16,772 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:16,772 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:16,772 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:16,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:16,772 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:17,021 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,021 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:17,021 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:17,021 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:17,021 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-11 00:54:17,021 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:17,021 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:17,021 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:17,021 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:17,022 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:17,023 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 00:54:17,023 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:17,024 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:17,024 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:17,024 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:17,024 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,025 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:17,025 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,025 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,025 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,026 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:17,026 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,026 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,026 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:17,027 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,027 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:17,027 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,027 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,027 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,027 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:17,027 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:17,028 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:17,028 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076887028 2023-02-11 00:54:17,029 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:17,030 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,032 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:17,033 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:17,033 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:17,033 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076917033 2023-02-11 00:54:17,033 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076977033 2023-02-11 00:54:17,033 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:17,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:17,035 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:17,035 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:17,035 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:17,283 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,283 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:17,283 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:17,283 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:17,284 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:17,284 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:17,284 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:17,284 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:17,284 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:17,285 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:17,286 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 00:54:17,286 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:17,286 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:17,287 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:17,287 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:17,287 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,287 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:17,288 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,288 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,288 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,288 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:17,289 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,289 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,289 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:17,289 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,289 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:17,289 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,290 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,290 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,290 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:17,290 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:17,291 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:17,291 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076887291 2023-02-11 00:54:17,291 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:17,293 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,295 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:17,295 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:17,296 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:17,296 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076917296 2023-02-11 00:54:17,296 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076977296 2023-02-11 00:54:17,296 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 00:54:17,296 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:17,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN}] 2023-02-11 00:54:17,298 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:17,298 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:17,298 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:17,546 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,546 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:17,546 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:17,547 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:17,548 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:17,548 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:17,548 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:17,548 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:17,548 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:17,549 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:17,550 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 00:54:17,550 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:17,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:17,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:17,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:17,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:17,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:17,554 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,554 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:17,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,555 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:17,555 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,555 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,555 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,556 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:54:17,556 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:17,557 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:17,557 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076887557 2023-02-11 00:54:17,559 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:17,559 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,561 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:17,561 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:17,562 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:17,562 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076917562 2023-02-11 00:54:17,562 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076977562 2023-02-11 00:54:17,562 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:54:17,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN}] 2023-02-11 00:54:17,562 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:17,564 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:17,565 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:17,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076857565"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076857565"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076857565"}]},"ts":"1676076857565"} 2023-02-11 00:54:17,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:17,566 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:17,566 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:17,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:17,566 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:17,812 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,812 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:17,812 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:17,812 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:17,814 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:17,814 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:17,814 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:17,814 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:17,814 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:17,815 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:17,816 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:17,816 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 00:54:17,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:17,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:17,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:17,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:17,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,819 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:17,819 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,819 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:17,820 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,820 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:17,820 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,821 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:17,821 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:17,822 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,822 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:17,822 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:17,822 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,822 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,823 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:17,823 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:17,823 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:17,823 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 00:54:17,828 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:17,828 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:17,829 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1 to restore RIT 2023-02-11 00:54:17,832 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076887832 2023-02-11 00:54:17,833 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:17,833 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:17,836 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:17,836 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:17,836 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:17,837 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076917837 2023-02-11 00:54:17,837 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076977837 2023-02-11 00:54:17,837 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:17,837 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:17,837 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076857837"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076857837"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076857837"}]},"ts":"1676076857837"} 2023-02-11 00:54:17,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:17,839 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:17,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:17,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 517bceec69b190e0374f76671f87d5d1, disabling compactions & flushes 2023-02-11 00:54:17,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:17,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:17,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. after waiting 0 ms 2023-02-11 00:54:17,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:17,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 517bceec69b190e0374f76671f87d5d1 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 00:54:18,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/0ed284438bb14c6db1ebcc439c62471a 2023-02-11 00:54:18,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/fe8bdb854e2941ca8a03b424006c21ef 2023-02-11 00:54:18,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/0ed284438bb14c6db1ebcc439c62471a as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a 2023-02-11 00:54:18,030 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:54:18,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/fe8bdb854e2941ca8a03b424006c21ef as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef 2023-02-11 00:54:18,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:54:18,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 517bceec69b190e0374f76671f87d5d1 in 42ms, sequenceid=82, compaction requested=true 2023-02-11 00:54:18,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 00:54:18,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:18,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:18,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 517bceec69b190e0374f76671f87d5d1: Waiting for close lock at 1676076857994Running coprocessor pre-close hooks at 1676076857994Disabling compacts and flushes for region at 1676076857994Disabling writes for close at 1676076857994Obtaining lock to block concurrent updates at 1676076857994Preparing flush snapshotting stores in 517bceec69b190e0374f76671f87d5d1 at 1676076857994Finished memstore snapshotting testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676076857995 (+1 ms)Flushing stores of testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. at 1676076857995Flushing 517bceec69b190e0374f76671f87d5d1/cf1: creating writer at 1676076857996 (+1 ms)Flushing 517bceec69b190e0374f76671f87d5d1/cf1: appending metadata at 1676076857998 (+2 ms)Flushing 517bceec69b190e0374f76671f87d5d1/cf1: closing flushed file at 1676076857998Flushing 517bceec69b190e0374f76671f87d5d1/cf2: creating writer at 1676076858013 (+15 ms)Flushing 517bceec69b190e0374f76671f87d5d1/cf2: appending metadata at 1676076858014 (+1 ms)Flushing 517bceec69b190e0374f76671f87d5d1/cf2: closing flushed file at 1676076858014Flushing 517bceec69b190e0374f76671f87d5d1/cf1: reopening flushed file at 1676076858027 (+13 ms)Flushing 517bceec69b190e0374f76671f87d5d1/cf2: reopening flushed file at 1676076858032 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 517bceec69b190e0374f76671f87d5d1 in 42ms, sequenceid=82, compaction requested=true at 1676076858036 (+4 ms)Writing region close event to WAL at 1676076858038 (+2 ms)Running coprocessor post-close hooks at 1676076858042 (+4 ms)Closed at 1676076858042 2023-02-11 00:54:18,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED 2023-02-11 00:54:18,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076858045"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076858045"}]},"ts":"1676076858045"} 2023-02-11 00:54:18,046 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:18,047 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:18,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:18,047 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:18,087 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,087 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:18,087 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:18,087 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:18,090 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:18,090 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:18,090 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:18,090 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:18,090 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:18,091 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:18,091 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:18,092 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 00:54:18,092 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-11 00:54:18,092 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:18,092 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:18,093 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:18,093 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:18,093 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,093 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:18,094 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,094 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,094 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,094 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:18,095 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,095 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,095 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:18,096 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,096 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:18,096 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:18,096 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:18,097 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,097 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,097 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,097 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:18,097 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:18,097 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:18,097 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:54:18,098 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:18,098 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:18,098 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1 to restore RIT 2023-02-11 00:54:18,099 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076888099 2023-02-11 00:54:18,100 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:18,100 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:18,102 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:18,103 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:18,103 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:18,103 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076918103 2023-02-11 00:54:18,103 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076978103 2023-02-11 00:54:18,103 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:18,103 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:18,103 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076858103"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076858103"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076858103"}]},"ts":"1676076858103"} 2023-02-11 00:54:18,105 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:18,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-11 00:54:18,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 in 266 msec 2023-02-11 00:54:18,107 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:18,107 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:18,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:18,108 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:18,353 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,353 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:18,353 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:18,353 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:18,354 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:18,354 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:18,355 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:18,355 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:18,355 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:18,355 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:18,356 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:18,356 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 00:54:18,356 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:18,357 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:18,357 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:18,357 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:18,358 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:18,358 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,358 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:18,358 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,358 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,359 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,359 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:18,359 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,359 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,359 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:18,360 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,360 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:18,360 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:18,360 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:18,360 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,361 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,361 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,361 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:18,361 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:18,361 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 00:54:18,361 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:18,362 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:18,362 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1 to restore RIT 2023-02-11 00:54:18,362 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076888362 2023-02-11 00:54:18,362 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:18,363 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:18,365 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:18,366 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:18,366 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:18,366 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076918366 2023-02-11 00:54:18,366 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076978366 2023-02-11 00:54:18,366 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:18,366 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:18,368 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:18,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-11 00:54:18,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN in 806 msec 2023-02-11 00:54:18,374 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:18,374 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:18,374 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:18,616 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,616 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:18,616 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:18,616 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:18,617 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:18,618 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:18,618 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:18,618 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:18,618 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:18,618 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:18,619 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:18,619 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:18,619 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:18,620 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:18,620 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:18,620 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:18,620 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:18,620 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,621 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:18,621 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,621 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:18,621 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,622 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:18,622 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,622 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:18,622 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:18,623 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,623 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:18,623 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:18,623 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:18,623 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:18,623 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:18,623 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:18,624 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:18,624 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:18,624 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076888624 2023-02-11 00:54:18,625 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:18,626 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,628 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:18,628 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:18,628 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:18,628 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076918628 2023-02-11 00:54:18,628 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076978628 2023-02-11 00:54:18,629 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:18,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,633 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:18,641 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=517bceec69b190e0374f76671f87d5d1, threads=14 2023-02-11 00:54:18,642 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,642 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,642 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,642 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,642 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,643 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,643 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,643 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,643 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,643 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,643 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,644 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,644 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,644 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,813 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,845 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,856 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,857 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,857 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,868 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,870 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,876 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,877 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,880 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,884 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,888 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,891 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,892 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:18,892 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 517bceec69b190e0374f76671f87d5d1 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:54:18,933 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:18,933 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:18,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:18,933 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:19,129 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,129 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:19,129 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:19,129 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:19,130 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:19,130 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:19,130 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:19,130 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:19,130 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:19,131 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:19,132 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 00:54:19,132 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:19,133 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:19,133 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:19,133 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:19,134 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:19,134 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:19,134 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:19,135 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:19,135 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:19,135 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:19,135 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:19,136 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:19,136 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:19,136 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:19,136 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:19,137 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,137 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:19,137 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:19,137 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:19,137 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,137 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,137 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,137 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:19,138 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:19,138 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:54:19,138 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:19,139 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:19,139 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076889139 2023-02-11 00:54:19,141 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:19,141 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,143 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:19,143 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:19,143 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:19,143 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076919143 2023-02-11 00:54:19,143 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076979143 2023-02-11 00:54:19,143 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:54:19,151 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=517bceec69b190e0374f76671f87d5d1, threads=14 2023-02-11 00:54:19,152 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,152 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,152 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,152 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,153 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,153 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,153 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,153 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,153 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,153 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,153 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,153 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,154 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,154 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,280 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,287 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,296 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,318 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,324 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,336 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,346 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,346 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,347 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,347 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,348 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,350 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,351 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 00:54:19,352 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,603 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55 for region: 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:19,603 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 517bceec69b190e0374f76671f87d5d1 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:54:19,645 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:19,650 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/ef64b99a250a430b37eb8cd170d91141/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:54:19,651 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/b07e9e94539895eecfacfe698f4f45f1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:54:19,651 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:19,651 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:19,651 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:19,894 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,894 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:19,894 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:19,894 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:19,895 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:19,895 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:19,895 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:19,895 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:19,895 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:19,896 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:19,897 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 00:54:19,897 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:19,897 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:19,897 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:19,897 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:19,897 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:19,898 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:19,898 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:19,898 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:19,898 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:19,898 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:19,899 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:19,899 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:19,899 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:19,899 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:19,899 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:19,900 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,900 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:19,900 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:19,902 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:19,902 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,902 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,902 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,902 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:19,902 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:19,902 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:54:19,903 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:19,903 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:19,904 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076889904 2023-02-11 00:54:19,905 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:19,905 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,907 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:19,907 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:19,907 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:19,907 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076919907 2023-02-11 00:54:19,907 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076979907 2023-02-11 00:54:19,907 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:54:19,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:19,913 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:19,913 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:19,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:19,913 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:20,158 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,158 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:20,158 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:20,158 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:20,159 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:20,159 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:20,159 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:20,160 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:20,160 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:20,160 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:20,161 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 00:54:20,161 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:20,161 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:20,161 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:20,162 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:20,162 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:20,162 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:20,162 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,163 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:20,163 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,163 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,163 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,164 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:20,164 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,164 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,164 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:20,164 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,165 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:20,165 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:20,165 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:20,165 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,165 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,165 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,165 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:20,165 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:20,165 DEBUG [Listener at localhost/35701] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,165 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:20,166 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:20,166 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:20,167 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076890166 2023-02-11 00:54:20,167 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:20,169 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,171 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:20,171 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:20,171 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:20,171 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076920171 2023-02-11 00:54:20,171 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076980171 2023-02-11 00:54:20,171 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:20,172 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:20,188 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:20,188 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 00:54:20,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:20,189 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:20,422 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,423 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:20,423 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:20,423 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:20,423 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:20,424 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:20,424 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:20,424 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:20,424 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:20,424 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:20,426 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-11 00:54:20,426 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:20,426 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:20,426 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:20,427 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:20,427 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:20,427 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:20,428 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,428 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:20,428 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,428 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,429 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,429 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:20,429 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,429 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,430 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:20,431 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,431 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:20,431 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:20,432 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:20,432 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,432 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,432 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,432 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:20,432 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:20,432 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 00:54:20,433 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:20,434 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:20,434 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076890434 2023-02-11 00:54:20,436 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:20,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,439 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:20,440 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:20,440 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:20,440 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076920440 2023-02-11 00:54:20,440 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076980440 2023-02-11 00:54:20,440 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:20,440 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:20,442 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:20,442 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 00:54:20,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:20,442 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:20,691 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,691 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:20,691 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:20,691 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:20,692 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:20,692 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:20,692 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:20,692 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:20,692 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:20,693 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:20,694 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 00:54:20,694 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:20,694 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:20,694 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:20,695 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:20,695 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:20,695 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:20,696 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,696 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:20,696 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,696 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,697 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,697 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:20,697 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,697 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,698 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:20,699 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,699 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:20,699 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:20,699 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:20,699 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,699 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,699 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,699 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:20,700 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:20,700 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:54:20,700 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:20,701 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:20,701 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076890701 2023-02-11 00:54:20,702 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:20,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,705 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:20,705 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:20,705 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:20,705 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076920705 2023-02-11 00:54:20,706 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076980706 2023-02-11 00:54:20,706 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:20,706 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:20,708 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:20,708 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 00:54:20,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:20,708 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:20,956 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,956 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:20,956 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:20,956 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:20,957 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:20,957 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:20,957 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:20,957 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:20,957 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:20,958 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:20,959 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 00:54:20,959 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:20,959 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:20,959 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:20,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:20,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:20,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:20,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:20,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:20,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:20,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:20,962 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:20,962 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,962 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:20,962 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:20,962 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:20,962 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,962 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,963 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,963 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:20,963 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:20,963 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:20,963 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:20,964 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:20,964 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076890964 2023-02-11 00:54:20,965 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:20,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:20,968 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:20,968 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:20,968 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:20,968 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076920968 2023-02-11 00:54:20,968 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076980968 2023-02-11 00:54:20,968 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:54:20,968 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:20,972 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN] 2023-02-11 00:54:20,972 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:20,972 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 00:54:20,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:20,972 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:21,218 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,219 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:21,219 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:21,219 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:21,219 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:21,219 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:21,219 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:21,219 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:21,219 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:21,220 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:21,221 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 00:54:21,221 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:54:21,221 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:54:21,221 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 00:54:21,221 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:21,221 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:21,222 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:21,222 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:21,222 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,222 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:21,222 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,222 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,223 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,223 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:21,223 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,223 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,223 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:21,224 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,224 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:21,224 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:21,225 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:21,225 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:21,225 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,225 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,225 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,225 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:21,225 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:21,225 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:21,225 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 00:54:21,226 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:21,226 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:21,226 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1 to restore RIT 2023-02-11 00:54:21,226 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076891226 2023-02-11 00:54:21,227 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:21,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,229 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:21,231 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:21,231 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:21,231 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:21,231 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076921231 2023-02-11 00:54:21,231 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076981231 2023-02-11 00:54:21,231 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 00:54:21,232 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:21,236 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:21,236 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 00:54:21,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:21,237 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:21,482 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,482 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:21,482 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:21,482 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:21,484 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:21,484 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:21,484 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:21,485 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:21,485 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:21,485 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:21,486 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:54:21,486 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 00:54:21,486 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:21,487 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:21,487 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:21,487 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:21,487 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,487 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:21,488 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,488 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,488 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,488 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:21,488 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,488 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,489 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:21,489 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,489 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:21,489 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:21,489 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,489 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,489 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,489 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:21,489 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:21,490 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:21,491 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:21,491 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1 to restore RIT 2023-02-11 00:54:21,491 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076891491 2023-02-11 00:54:21,493 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:21,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,494 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:21,496 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:21,496 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:21,496 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:21,496 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076921496 2023-02-11 00:54:21,496 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076981496 2023-02-11 00:54:21,496 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:21,496 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:21,499 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:21,499 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 00:54:21,499 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:21,499 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:21,747 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,747 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:21,747 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:21,747 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:21,747 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:21,747 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:21,747 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:21,747 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:21,747 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:21,748 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:21,749 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 00:54:21,749 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 00:54:21,749 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:21,749 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:21,749 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:21,750 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:21,750 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,750 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:21,750 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,750 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:21,751 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,751 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:21,751 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,751 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:21,751 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:21,752 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,752 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:21,752 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:21,752 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,752 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,752 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,752 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:21,752 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 00:54:21,753 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:21,753 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:21,753 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1 to restore RIT 2023-02-11 00:54:21,754 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076891754 2023-02-11 00:54:21,755 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:21,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:21,756 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:21,758 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:21,758 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:21,758 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:21,758 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076921758 2023-02-11 00:54:21,758 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076981758 2023-02-11 00:54:21,759 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:54:21,759 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:21,761 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:21,761 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 00:54:21,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:21,761 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:22,009 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:22,009 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:22,009 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:22,009 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:22,009 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:22,009 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:22,009 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:22,009 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:22,010 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:22,010 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:22,011 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 00:54:22,011 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 00:54:22,011 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:22,012 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:22,012 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:22,012 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:22,012 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:22,013 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:22,013 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:22,013 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:22,014 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:22,014 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:22,014 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:22,014 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:22,015 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:22,015 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:22,015 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:22,015 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:22,016 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:22,016 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:22,016 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:22,016 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:54:22,016 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 00:54:22,017 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:22,017 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:22,017 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1 to restore RIT 2023-02-11 00:54:22,018 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076892017 2023-02-11 00:54:22,019 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:22,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:22,022 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:22,024 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:22,024 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:22,024 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:22,024 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076922024 2023-02-11 00:54:22,024 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076982024 2023-02-11 00:54:22,024 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 00:54:22,024 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:54:22,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 exec-time=5.2550 sec 2023-02-11 00:54:22,027 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:22,028 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,33641,1676076840386; forceNewPlan=false, retain=true 2023-02-11 00:54:22,178 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:22,179 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:22,179 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076862179"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076862179"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076862179"}]},"ts":"1676076862179"} 2023-02-11 00:54:22,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:22,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:54:22,181 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:22,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:54:22,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:54:22,275 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:22,275 WARN [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-11 00:54:22,275 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:54:22,275 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:54:22,275 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:54:22,275 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:54:22,275 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:54:22,275 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:54:22,275 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:54:22,275 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:54:22,276 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:54:22,277 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-11 00:54:22,277 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:54:22,277 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:54:22,277 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:54:22,278 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:54:22,278 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:22,278 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:54:22,278 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:22,278 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:54:22,278 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:22,279 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:54:22,279 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:22,279 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:54:22,279 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:22,279 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:54:22,279 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:22,280 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:54:22,280 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:22,280 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-11 00:54:22,280 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:54:22,281 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:54:22,281 INFO [Listener at localhost/35701] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1 to restore RIT 2023-02-11 00:54:22,281 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076892281 2023-02-11 00:54:22,282 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:54:22,282 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:54:22,284 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:54:22,284 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=517bceec69b190e0374f76671f87d5d1, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,33641,1676076840386, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386, openSeqNum=2 2023-02-11 00:54:22,284 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:54:22,284 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076922284 2023-02-11 00:54:22,284 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676076982284 2023-02-11 00:54:22,284 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:54:22,285 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:22,285 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076862285"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076862285"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076862285"}]},"ts":"1676076862285"} 2023-02-11 00:54:22,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:22,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:22,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 517bceec69b190e0374f76671f87d5d1, NAME => 'testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:22,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:22,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:22,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:22,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:22,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:22,444 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:22,444 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1 2023-02-11 00:54:22,445 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1 2023-02-11 00:54:22,445 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 517bceec69b190e0374f76671f87d5d1 columnFamilyName cf1 2023-02-11 00:54:22,452 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0 2023-02-11 00:54:22,455 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a 2023-02-11 00:54:22,457 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64 2023-02-11 00:54:22,460 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532 2023-02-11 00:54:22,463 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b 2023-02-11 00:54:22,466 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55 2023-02-11 00:54:22,470 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f 2023-02-11 00:54:22,470 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(310): Store=517bceec69b190e0374f76671f87d5d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:22,470 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:22,471 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2 2023-02-11 00:54:22,471 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2 2023-02-11 00:54:22,471 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 517bceec69b190e0374f76671f87d5d1 columnFamilyName cf2 2023-02-11 00:54:22,477 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c 2023-02-11 00:54:22,481 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7 2023-02-11 00:54:22,484 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0 2023-02-11 00:54:22,487 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1 2023-02-11 00:54:22,490 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285 2023-02-11 00:54:22,493 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0 2023-02-11 00:54:22,496 DEBUG [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef 2023-02-11 00:54:22,496 INFO [StoreOpener-517bceec69b190e0374f76671f87d5d1-1] regionserver.HStore(310): Store=517bceec69b190e0374f76671f87d5d1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:22,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:22,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:22,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.splits 2023-02-11 00:54:22,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:22,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:22,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 517bceec69b190e0374f76671f87d5d1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70433829, jitterRate=0.04954583942890167}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:22,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 517bceec69b190e0374f76671f87d5d1: Running coprocessor pre-open hook at 1676076862443Writing region info on filesystem at 1676076862443Initializing all the Stores at 1676076862443Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076862443Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076862444 (+1 ms)Cleaning up temporary data from old regions at 1676076862498 (+54 ms)Cleaning up detritus from prior splits at 1676076862498Running coprocessor post-open hooks at 1676076862501 (+3 ms)Region opened successfully at 1676076862501 2023-02-11 00:54:22,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1., pid=48, masterSystemTime=1676076862438 2023-02-11 00:54:22,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:22,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:54:22,503 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:22,503 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:22,504 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 00:54:22,504 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 00:54:22,504 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1912): 517bceec69b190e0374f76671f87d5d1/cf1 is initiating minor compaction (all files) 2023-02-11 00:54:22,504 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1912): 517bceec69b190e0374f76671f87d5d1/cf2 is initiating minor compaction (all files) 2023-02-11 00:54:22,504 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 517bceec69b190e0374f76671f87d5d1/cf1 in testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:22,504 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 517bceec69b190e0374f76671f87d5d1/cf2 in testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:22,505 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp, totalSize=34.9 K 2023-02-11 00:54:22,505 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp, totalSize=34.9 K 2023-02-11 00:54:22,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:22,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:22,506 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting c44cbfe5d17a400abb72cb0836c08d55, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076855380 2023-02-11 00:54:22,506 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting d2282319e6514486adb4b895349ff1f0, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076855380 2023-02-11 00:54:22,506 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:22,506 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 098ac1c41a8d40e0b1193154f0a4eab0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076855571 2023-02-11 00:54:22,506 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076862506"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076862506"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076862506"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076862506"}]},"ts":"1676076862506"} 2023-02-11 00:54:22,506 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 3bc9b70646ae422a9394a219a645edc7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076855571 2023-02-11 00:54:22,507 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 9db8268ca1ce4147b943228156b6269b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076855756 2023-02-11 00:54:22,507 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 4609fa2424e5493a82109ee1e9e7d7b0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076855756 2023-02-11 00:54:22,507 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting d1448290ffc64f938c9a94a5c20cba5f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076855949 2023-02-11 00:54:22,507 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 07ce53397b494ba49b5976a67c13795c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076855949 2023-02-11 00:54:22,508 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 4e27c584057b4b4c88727a838f12d532, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076856144 2023-02-11 00:54:22,508 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 634d93347ec74d2c837f25e586834df1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076856144 2023-02-11 00:54:22,508 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 10d8c709e2234bce84e59f274912fd64, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076856325 2023-02-11 00:54:22,508 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting b835a27056274355bd1dccc95d5fa285, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076856325 2023-02-11 00:54:22,509 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 0ed284438bb14c6db1ebcc439c62471a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076856509 2023-02-11 00:54:22,509 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting fe8bdb854e2941ca8a03b424006c21ef, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076856509 2023-02-11 00:54:22,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-11 00:54:22,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 in 222 msec 2023-02-11 00:54:22,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN in 1.5410 sec 2023-02-11 00:54:22,532 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] throttle.PressureAwareThroughputController(145): 517bceec69b190e0374f76671f87d5d1#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:22,532 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] throttle.PressureAwareThroughputController(145): 517bceec69b190e0374f76671f87d5d1#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:22,545 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf1/ba12953e2e104f90ba24e37ee17d68af as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/ba12953e2e104f90ba24e37ee17d68af 2023-02-11 00:54:22,550 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 517bceec69b190e0374f76671f87d5d1/cf1 of 517bceec69b190e0374f76671f87d5d1 into ba12953e2e104f90ba24e37ee17d68af(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:22,550 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 517bceec69b190e0374f76671f87d5d1: 2023-02-11 00:54:22,550 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1., storeName=517bceec69b190e0374f76671f87d5d1/cf1, priority=9, startTime=1676076862502; duration=0sec 2023-02-11 00:54:22,551 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:22,861 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-11 00:54:22,864 DEBUG [Listener at localhost/35701] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-11 00:54:22,866 INFO [Listener at localhost/35701] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-11 00:54:22,868 INFO [Listener at localhost/35701] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-11 00:54:22,869 INFO [Listener at localhost/35701] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-11 00:54:22,869 INFO [Listener at localhost/35701] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-11 00:54:22,899 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:22,901 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-11 00:54:22,902 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-11 00:54:22,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:22,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 00:54:22,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076862905"}]},"ts":"1676076862905"} 2023-02-11 00:54:22,907 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-11 00:54:22,908 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-11 00:54:22,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN}] 2023-02-11 00:54:22,910 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN 2023-02-11 00:54:22,910 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:22,911 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076862910"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076862910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076862910"}]},"ts":"1676076862910"} 2023-02-11 00:54:22,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:22,949 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/.tmp/cf2/30e8941f2084461296dfb780cb3c0a3c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/30e8941f2084461296dfb780cb3c0a3c 2023-02-11 00:54:22,953 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 517bceec69b190e0374f76671f87d5d1/cf2 of 517bceec69b190e0374f76671f87d5d1 into 30e8941f2084461296dfb780cb3c0a3c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:22,953 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 517bceec69b190e0374f76671f87d5d1: 2023-02-11 00:54:22,953 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1., storeName=517bceec69b190e0374f76671f87d5d1/cf2, priority=9, startTime=1676076862502; duration=0sec 2023-02-11 00:54:22,953 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:23,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 00:54:23,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:23,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 517bceec69b190e0374f76671f87d5d1, disabling compactions & flushes 2023-02-11 00:54:23,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:23,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:23,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. after waiting 0 ms 2023-02-11 00:54:23,064 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:23,065 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a] to archive 2023-02-11 00:54:23,065 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:23,067 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/c44cbfe5d17a400abb72cb0836c08d55 2023-02-11 00:54:23,069 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/098ac1c41a8d40e0b1193154f0a4eab0 2023-02-11 00:54:23,070 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/9db8268ca1ce4147b943228156b6269b 2023-02-11 00:54:23,071 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/d1448290ffc64f938c9a94a5c20cba5f 2023-02-11 00:54:23,072 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/4e27c584057b4b4c88727a838f12d532 2023-02-11 00:54:23,074 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/10d8c709e2234bce84e59f274912fd64 2023-02-11 00:54:23,075 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/0ed284438bb14c6db1ebcc439c62471a 2023-02-11 00:54:23,079 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef] to archive 2023-02-11 00:54:23,079 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:23,081 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/d2282319e6514486adb4b895349ff1f0 2023-02-11 00:54:23,082 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/3bc9b70646ae422a9394a219a645edc7 2023-02-11 00:54:23,083 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/4609fa2424e5493a82109ee1e9e7d7b0 2023-02-11 00:54:23,084 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/07ce53397b494ba49b5976a67c13795c 2023-02-11 00:54:23,085 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/634d93347ec74d2c837f25e586834df1 2023-02-11 00:54:23,086 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/b835a27056274355bd1dccc95d5fa285 2023-02-11 00:54:23,086 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/fe8bdb854e2941ca8a03b424006c21ef 2023-02-11 00:54:23,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:54:23,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:23,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1. 2023-02-11 00:54:23,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 517bceec69b190e0374f76671f87d5d1: Waiting for close lock at 1676076863064Running coprocessor pre-close hooks at 1676076863064Disabling compacts and flushes for region at 1676076863064Disabling writes for close at 1676076863064Writing region close event to WAL at 1676076863087 (+23 ms)Running coprocessor post-close hooks at 1676076863091 (+4 ms)Closed at 1676076863091 2023-02-11 00:54:23,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:23,093 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=517bceec69b190e0374f76671f87d5d1, regionState=CLOSED 2023-02-11 00:54:23,093 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076863093"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076863093"}]},"ts":"1676076863093"} 2023-02-11 00:54:23,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-11 00:54:23,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 517bceec69b190e0374f76671f87d5d1, server=jenkins-hbase4.apache.org,33641,1676076840386 in 182 msec 2023-02-11 00:54:23,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-11 00:54:23,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, UNASSIGN in 187 msec 2023-02-11 00:54:23,098 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076863098"}]},"ts":"1676076863098"} 2023-02-11 00:54:23,099 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-11 00:54:23,100 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-11 00:54:23,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 199 msec 2023-02-11 00:54:23,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 00:54:23,208 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-11 00:54:23,208 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-11 00:54:23,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:23,211 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:23,212 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:23,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-11 00:54:23,215 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:23,217 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/recovered.edits] 2023-02-11 00:54:23,219 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/ba12953e2e104f90ba24e37ee17d68af to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf1/ba12953e2e104f90ba24e37ee17d68af 2023-02-11 00:54:23,221 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/30e8941f2084461296dfb780cb3c0a3c to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/cf2/30e8941f2084461296dfb780cb3c0a3c 2023-02-11 00:54:23,223 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1/recovered.edits/90.seqid 2023-02-11 00:54:23,224 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackAndDoubleExecution/517bceec69b190e0374f76671f87d5d1 2023-02-11 00:54:23,224 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-11 00:54:23,227 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:23,228 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-11 00:54:23,230 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-11 00:54:23,231 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:23,231 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-11 00:54:23,231 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076863231"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:23,233 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 00:54:23,233 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 517bceec69b190e0374f76671f87d5d1, NAME => 'testRollbackAndDoubleExecution,,1676076854876.517bceec69b190e0374f76671f87d5d1.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 00:54:23,233 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-11 00:54:23,233 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076863233"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:23,235 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-11 00:54:23,237 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:54:23,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 29 msec 2023-02-11 00:54:23,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-11 00:54:23,314 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-11 00:54:23,328 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400 (was 399) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51894 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51932 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33876 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:33641-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51958 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33904 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33890 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33934 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:40950 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51942 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51890 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33916 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33912 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51888 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51906 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51914 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51920 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=869 (was 858) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=193 (was 209), ProcessCount=168 (was 168), AvailableMemoryMB=3260 (was 3277) 2023-02-11 00:54:23,342 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=400, OpenFileDescriptor=869, MaxFileDescriptor=60000, SystemLoadAverage=193, ProcessCount=168, AvailableMemoryMB=3260 2023-02-11 00:54:23,342 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:23,346 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:54:23,347 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:54:23,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:54:23,349 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:54:23,350 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:54:23,351 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,352 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1 empty. 2023-02-11 00:54:23,352 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,352 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-11 00:54:23,363 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:23,364 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2645d4c46317f3690a043a36a348b9c1, NAME => 'testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:54:23,380 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:23,380 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 2645d4c46317f3690a043a36a348b9c1, disabling compactions & flushes 2023-02-11 00:54:23,380 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:23,380 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:23,380 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. after waiting 0 ms 2023-02-11 00:54:23,380 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:23,380 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:23,380 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 2645d4c46317f3690a043a36a348b9c1: Waiting for close lock at 1676076863380Disabling compacts and flushes for region at 1676076863380Disabling writes for close at 1676076863380Writing region close event to WAL at 1676076863380Closed at 1676076863380 2023-02-11 00:54:23,383 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:54:23,384 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676076863383"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076863383"}]},"ts":"1676076863383"} 2023-02-11 00:54:23,385 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:54:23,386 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:54:23,386 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076863386"}]},"ts":"1676076863386"} 2023-02-11 00:54:23,387 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-11 00:54:23,390 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:23,391 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:23,391 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:23,391 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:23,391 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:54:23,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2645d4c46317f3690a043a36a348b9c1, ASSIGN}] 2023-02-11 00:54:23,393 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2645d4c46317f3690a043a36a348b9c1, ASSIGN 2023-02-11 00:54:23,394 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2645d4c46317f3690a043a36a348b9c1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:23,544 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:23,545 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=2645d4c46317f3690a043a36a348b9c1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:23,546 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676076863545"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076863545"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076863545"}]},"ts":"1676076863545"} 2023-02-11 00:54:23,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 2645d4c46317f3690a043a36a348b9c1, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:23,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:23,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2645d4c46317f3690a043a36a348b9c1, NAME => 'testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:23,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:23,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:23,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,703 INFO [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,705 DEBUG [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/cf1 2023-02-11 00:54:23,705 DEBUG [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/cf1 2023-02-11 00:54:23,705 INFO [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2645d4c46317f3690a043a36a348b9c1 columnFamilyName cf1 2023-02-11 00:54:23,706 INFO [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] regionserver.HStore(310): Store=2645d4c46317f3690a043a36a348b9c1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:23,706 INFO [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,707 DEBUG [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/cf2 2023-02-11 00:54:23,707 DEBUG [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/cf2 2023-02-11 00:54:23,708 INFO [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2645d4c46317f3690a043a36a348b9c1 columnFamilyName cf2 2023-02-11 00:54:23,708 INFO [StoreOpener-2645d4c46317f3690a043a36a348b9c1-1] regionserver.HStore(310): Store=2645d4c46317f3690a043a36a348b9c1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:23,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:23,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:23,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:23,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2645d4c46317f3690a043a36a348b9c1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58793201, jitterRate=-0.12391303479671478}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:23,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2645d4c46317f3690a043a36a348b9c1: Running coprocessor pre-open hook at 1676076863702Writing region info on filesystem at 1676076863702Initializing all the Stores at 1676076863703 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076863703Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076863703Cleaning up temporary data from old regions at 1676076863710 (+7 ms)Cleaning up detritus from prior splits at 1676076863710Running coprocessor post-open hooks at 1676076863715 (+5 ms)Region opened successfully at 1676076863715 2023-02-11 00:54:23,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1., pid=55, masterSystemTime=1676076863699 2023-02-11 00:54:23,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:23,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:23,718 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=2645d4c46317f3690a043a36a348b9c1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:23,718 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676076863718"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076863718"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076863718"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076863718"}]},"ts":"1676076863718"} 2023-02-11 00:54:23,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-11 00:54:23,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 2645d4c46317f3690a043a36a348b9c1, server=jenkins-hbase4.apache.org,38317,1676076840434 in 172 msec 2023-02-11 00:54:23,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-11 00:54:23,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2645d4c46317f3690a043a36a348b9c1, ASSIGN in 330 msec 2023-02-11 00:54:23,723 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:54:23,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076863723"}]},"ts":"1676076863723"} 2023-02-11 00:54:23,725 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-11 00:54:23,726 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:54:23,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 379 msec 2023-02-11 00:54:23,849 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:23,850 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:23,851 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:23,851 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:23,851 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:23,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2645d4c46317f3690a043a36a348b9c1, UNASSIGN}] 2023-02-11 00:54:23,854 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2645d4c46317f3690a043a36a348b9c1, UNASSIGN 2023-02-11 00:54:23,854 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=2645d4c46317f3690a043a36a348b9c1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:23,855 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676076863854"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076863854"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076863854"}]},"ts":"1676076863854"} 2023-02-11 00:54:23,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 2645d4c46317f3690a043a36a348b9c1, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:24,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:24,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2645d4c46317f3690a043a36a348b9c1, disabling compactions & flushes 2023-02-11 00:54:24,009 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:24,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:24,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. after waiting 0 ms 2023-02-11 00:54:24,009 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:24,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-11 00:54:24,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:24,013 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1. 2023-02-11 00:54:24,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2645d4c46317f3690a043a36a348b9c1: Waiting for close lock at 1676076864008Running coprocessor pre-close hooks at 1676076864008Disabling compacts and flushes for region at 1676076864008Disabling writes for close at 1676076864009 (+1 ms)Writing region close event to WAL at 1676076864009Running coprocessor post-close hooks at 1676076864013 (+4 ms)Closed at 1676076864013 2023-02-11 00:54:24,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:24,015 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=2645d4c46317f3690a043a36a348b9c1, regionState=CLOSED 2023-02-11 00:54:24,015 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676076864015"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076864015"}]},"ts":"1676076864015"} 2023-02-11 00:54:24,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-11 00:54:24,018 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 2645d4c46317f3690a043a36a348b9c1, server=jenkins-hbase4.apache.org,38317,1676076840434 in 161 msec 2023-02-11 00:54:24,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-11 00:54:24,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2645d4c46317f3690a043a36a348b9c1, UNASSIGN in 165 msec 2023-02-11 00:54:24,046 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-11 00:54:24,048 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-11 00:54:24,049 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676076864049"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676076864049"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676076864049"}]},"ts":"1676076864049"} 2023-02-11 00:54:24,049 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076864049"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076864049"}]},"ts":"1676076864049"} 2023-02-11 00:54:24,049 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864049"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076864049"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076864049"}]},"ts":"1676076864049"} 2023-02-11 00:54:24,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=95f542afca1500322e0c0b9924479821, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=639a8ff09cb3f0dc787eb4be08877b2e, ASSIGN}] 2023-02-11 00:54:24,053 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=639a8ff09cb3f0dc787eb4be08877b2e, ASSIGN 2023-02-11 00:54:24,054 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=639a8ff09cb3f0dc787eb4be08877b2e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:24,054 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=95f542afca1500322e0c0b9924479821, ASSIGN 2023-02-11 00:54:24,055 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=95f542afca1500322e0c0b9924479821, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:24,204 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 00:54:24,205 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=95f542afca1500322e0c0b9924479821, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:24,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864205"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076864205"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076864205"}]},"ts":"1676076864205"} 2023-02-11 00:54:24,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 95f542afca1500322e0c0b9924479821, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:24,210 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=639a8ff09cb3f0dc787eb4be08877b2e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:24,210 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864210"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076864210"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076864210"}]},"ts":"1676076864210"} 2023-02-11 00:54:24,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 639a8ff09cb3f0dc787eb4be08877b2e, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:24,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821. 2023-02-11 00:54:24,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 95f542afca1500322e0c0b9924479821, NAME => 'testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 00:54:24,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:24,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:24,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,363 INFO [StoreOpener-95f542afca1500322e0c0b9924479821-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,365 DEBUG [StoreOpener-95f542afca1500322e0c0b9924479821-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/cf1 2023-02-11 00:54:24,365 DEBUG [StoreOpener-95f542afca1500322e0c0b9924479821-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/cf1 2023-02-11 00:54:24,365 INFO [StoreOpener-95f542afca1500322e0c0b9924479821-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95f542afca1500322e0c0b9924479821 columnFamilyName cf1 2023-02-11 00:54:24,366 INFO [StoreOpener-95f542afca1500322e0c0b9924479821-1] regionserver.HStore(310): Store=95f542afca1500322e0c0b9924479821/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:24,366 INFO [StoreOpener-95f542afca1500322e0c0b9924479821-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,367 DEBUG [StoreOpener-95f542afca1500322e0c0b9924479821-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/cf2 2023-02-11 00:54:24,367 DEBUG [StoreOpener-95f542afca1500322e0c0b9924479821-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/cf2 2023-02-11 00:54:24,367 INFO [StoreOpener-95f542afca1500322e0c0b9924479821-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95f542afca1500322e0c0b9924479821 columnFamilyName cf2 2023-02-11 00:54:24,368 INFO [StoreOpener-95f542afca1500322e0c0b9924479821-1] regionserver.HStore(310): Store=95f542afca1500322e0c0b9924479821/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:24,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:24,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 95f542afca1500322e0c0b9924479821; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74873803, jitterRate=0.11570660769939423}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:24,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 95f542afca1500322e0c0b9924479821: Running coprocessor pre-open hook at 1676076864363Writing region info on filesystem at 1676076864363Initializing all the Stores at 1676076864363Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076864363Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076864363Cleaning up temporary data from old regions at 1676076864370 (+7 ms)Cleaning up detritus from prior splits at 1676076864370Running coprocessor post-open hooks at 1676076864373 (+3 ms)Region opened successfully at 1676076864373 2023-02-11 00:54:24,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821., pid=61, masterSystemTime=1676076864359 2023-02-11 00:54:24,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821. 2023-02-11 00:54:24,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821. 2023-02-11 00:54:24,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e. 2023-02-11 00:54:24,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 639a8ff09cb3f0dc787eb4be08877b2e, NAME => 'testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 00:54:24,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:24,375 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=95f542afca1500322e0c0b9924479821, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:24,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:24,375 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864375"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076864375"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076864375"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076864375"}]},"ts":"1676076864375"} 2023-02-11 00:54:24,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,376 INFO [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,377 DEBUG [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/cf1 2023-02-11 00:54:24,377 DEBUG [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/cf1 2023-02-11 00:54:24,378 INFO [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 639a8ff09cb3f0dc787eb4be08877b2e columnFamilyName cf1 2023-02-11 00:54:24,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-11 00:54:24,378 INFO [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] regionserver.HStore(310): Store=639a8ff09cb3f0dc787eb4be08877b2e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:24,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 95f542afca1500322e0c0b9924479821, server=jenkins-hbase4.apache.org,38317,1676076840434 in 169 msec 2023-02-11 00:54:24,378 INFO [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=95f542afca1500322e0c0b9924479821, ASSIGN in 327 msec 2023-02-11 00:54:24,380 DEBUG [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/cf2 2023-02-11 00:54:24,380 DEBUG [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/cf2 2023-02-11 00:54:24,380 INFO [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 639a8ff09cb3f0dc787eb4be08877b2e columnFamilyName cf2 2023-02-11 00:54:24,380 INFO [StoreOpener-639a8ff09cb3f0dc787eb4be08877b2e-1] regionserver.HStore(310): Store=639a8ff09cb3f0dc787eb4be08877b2e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:24,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:24,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 639a8ff09cb3f0dc787eb4be08877b2e; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75285001, jitterRate=0.121833935379982}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:24,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 639a8ff09cb3f0dc787eb4be08877b2e: Running coprocessor pre-open hook at 1676076864375Writing region info on filesystem at 1676076864375Initializing all the Stores at 1676076864376 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076864376Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076864376Cleaning up temporary data from old regions at 1676076864383 (+7 ms)Cleaning up detritus from prior splits at 1676076864383Running coprocessor post-open hooks at 1676076864385 (+2 ms)Region opened successfully at 1676076864385 2023-02-11 00:54:24,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e., pid=62, masterSystemTime=1676076864359 2023-02-11 00:54:24,387 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e. 2023-02-11 00:54:24,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e. 2023-02-11 00:54:24,388 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=639a8ff09cb3f0dc787eb4be08877b2e, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:24,388 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864387"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076864387"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076864387"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076864387"}]},"ts":"1676076864387"} 2023-02-11 00:54:24,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-11 00:54:24,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 639a8ff09cb3f0dc787eb4be08877b2e, server=jenkins-hbase4.apache.org,38317,1676076840434 in 178 msec 2023-02-11 00:54:24,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-02-11 00:54:24,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=639a8ff09cb3f0dc787eb4be08877b2e, ASSIGN in 338 msec 2023-02-11 00:54:24,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e in 543 msec 2023-02-11 00:54:24,604 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 00:54:24,607 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:24,609 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,609 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 00:54:24,613 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076864613"}]},"ts":"1676076864613"} 2023-02-11 00:54:24,614 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-11 00:54:24,616 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-11 00:54:24,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=95f542afca1500322e0c0b9924479821, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=639a8ff09cb3f0dc787eb4be08877b2e, UNASSIGN}] 2023-02-11 00:54:24,617 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=639a8ff09cb3f0dc787eb4be08877b2e, UNASSIGN 2023-02-11 00:54:24,617 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=639a8ff09cb3f0dc787eb4be08877b2e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:24,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864617"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076864617"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076864617"}]},"ts":"1676076864617"} 2023-02-11 00:54:24,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 639a8ff09cb3f0dc787eb4be08877b2e, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:24,621 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=95f542afca1500322e0c0b9924479821, UNASSIGN 2023-02-11 00:54:24,622 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=95f542afca1500322e0c0b9924479821, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:24,622 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864622"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076864622"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076864622"}]},"ts":"1676076864622"} 2023-02-11 00:54:24,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 95f542afca1500322e0c0b9924479821, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:24,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 00:54:24,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 95f542afca1500322e0c0b9924479821, disabling compactions & flushes 2023-02-11 00:54:24,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821. 2023-02-11 00:54:24,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821. 2023-02-11 00:54:24,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821. after waiting 0 ms 2023-02-11 00:54:24,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821. 2023-02-11 00:54:24,776 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 00:54:24,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:24,777 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821. 2023-02-11 00:54:24,777 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 95f542afca1500322e0c0b9924479821: Waiting for close lock at 1676076864771Running coprocessor pre-close hooks at 1676076864771Disabling compacts and flushes for region at 1676076864771Disabling writes for close at 1676076864772 (+1 ms)Writing region close event to WAL at 1676076864772Running coprocessor post-close hooks at 1676076864777 (+5 ms)Closed at 1676076864777 2023-02-11 00:54:24,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 639a8ff09cb3f0dc787eb4be08877b2e, disabling compactions & flushes 2023-02-11 00:54:24,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e. 2023-02-11 00:54:24,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e. 2023-02-11 00:54:24,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e. after waiting 0 ms 2023-02-11 00:54:24,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e. 2023-02-11 00:54:24,780 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=95f542afca1500322e0c0b9924479821, regionState=CLOSED 2023-02-11 00:54:24,780 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864780"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076864780"}]},"ts":"1676076864780"} 2023-02-11 00:54:24,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-11 00:54:24,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 95f542afca1500322e0c0b9924479821, server=jenkins-hbase4.apache.org,38317,1676076840434 in 159 msec 2023-02-11 00:54:24,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=95f542afca1500322e0c0b9924479821, UNASSIGN in 168 msec 2023-02-11 00:54:24,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 00:54:24,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:24,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e. 2023-02-11 00:54:24,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 639a8ff09cb3f0dc787eb4be08877b2e: Waiting for close lock at 1676076864779Running coprocessor pre-close hooks at 1676076864779Disabling compacts and flushes for region at 1676076864779Disabling writes for close at 1676076864780 (+1 ms)Writing region close event to WAL at 1676076864780Running coprocessor post-close hooks at 1676076864797 (+17 ms)Closed at 1676076864797 2023-02-11 00:54:24,799 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,799 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=639a8ff09cb3f0dc787eb4be08877b2e, regionState=CLOSED 2023-02-11 00:54:24,799 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076864799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076864799"}]},"ts":"1676076864799"} 2023-02-11 00:54:24,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-11 00:54:24,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 639a8ff09cb3f0dc787eb4be08877b2e, server=jenkins-hbase4.apache.org,38317,1676076840434 in 182 msec 2023-02-11 00:54:24,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-11 00:54:24,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=639a8ff09cb3f0dc787eb4be08877b2e, UNASSIGN in 187 msec 2023-02-11 00:54:24,805 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076864805"}]},"ts":"1676076864805"} 2023-02-11 00:54:24,806 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-11 00:54:24,808 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-11 00:54:24,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 199 msec 2023-02-11 00:54:24,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 00:54:24,915 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-11 00:54:24,916 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,918 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,919 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-11 00:54:24,924 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:24,924 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,924 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,926 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/recovered.edits] 2023-02-11 00:54:24,926 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/recovered.edits] 2023-02-11 00:54:24,926 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/recovered.edits] 2023-02-11 00:54:24,933 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/recovered.edits/7.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821/recovered.edits/7.seqid 2023-02-11 00:54:24,933 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/recovered.edits/7.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e/recovered.edits/7.seqid 2023-02-11 00:54:24,934 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/recovered.edits/4.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1/recovered.edits/4.seqid 2023-02-11 00:54:24,934 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/95f542afca1500322e0c0b9924479821 2023-02-11 00:54:24,934 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/2645d4c46317f3690a043a36a348b9c1 2023-02-11 00:54:24,934 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionNoStoreFile/639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:54:24,934 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-11 00:54:24,936 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,938 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-11 00:54:24,939 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-11 00:54:24,940 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,940 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-11 00:54:24,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076864940"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:24,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076864940"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:24,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076864940"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:24,942 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 00:54:24,942 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2645d4c46317f3690a043a36a348b9c1, NAME => 'testSplitTableRegionNoStoreFile,,1676076863347.2645d4c46317f3690a043a36a348b9c1.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 95f542afca1500322e0c0b9924479821, NAME => 'testSplitTableRegionNoStoreFile,,1676076863849.95f542afca1500322e0c0b9924479821.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 639a8ff09cb3f0dc787eb4be08877b2e, NAME => 'testSplitTableRegionNoStoreFile,41,1676076863849.639a8ff09cb3f0dc787eb4be08877b2e.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 00:54:24,942 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-11 00:54:24,942 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076864942"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:24,943 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-11 00:54:24,946 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:54:24,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 30 msec 2023-02-11 00:54:25,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-11 00:54:25,020 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-11 00:54:25,034 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=404 (was 400) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_715397967_15 at /127.0.0.1:33876 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_715397967_15 at /127.0.0.1:33890 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=869 (was 869), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=193 (was 193), ProcessCount=168 (was 168), AvailableMemoryMB=3258 (was 3260) 2023-02-11 00:54:25,048 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=404, OpenFileDescriptor=869, MaxFileDescriptor=60000, SystemLoadAverage=193, ProcessCount=168, AvailableMemoryMB=3257 2023-02-11 00:54:25,049 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:25,051 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:54:25,051 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:54:25,053 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,054 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:54:25,055 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:54:25,056 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,057 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35 empty. 2023-02-11 00:54:25,057 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,057 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-11 00:54:25,067 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:25,068 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => e3beb69538b330eb0b73628e03f86d35, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:54:25,075 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:25,075 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing e3beb69538b330eb0b73628e03f86d35, disabling compactions & flushes 2023-02-11 00:54:25,076 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,076 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,076 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. after waiting 0 ms 2023-02-11 00:54:25,076 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,076 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,076 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for e3beb69538b330eb0b73628e03f86d35: Waiting for close lock at 1676076865075Disabling compacts and flushes for region at 1676076865075Disabling writes for close at 1676076865076 (+1 ms)Writing region close event to WAL at 1676076865076Closed at 1676076865076 2023-02-11 00:54:25,077 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:54:25,078 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676076865078"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076865078"}]},"ts":"1676076865078"} 2023-02-11 00:54:25,080 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:54:25,080 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:54:25,081 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076865080"}]},"ts":"1676076865080"} 2023-02-11 00:54:25,081 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-11 00:54:25,084 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:25,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:25,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:25,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:25,085 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:54:25,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e3beb69538b330eb0b73628e03f86d35, ASSIGN}] 2023-02-11 00:54:25,087 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e3beb69538b330eb0b73628e03f86d35, ASSIGN 2023-02-11 00:54:25,087 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e3beb69538b330eb0b73628e03f86d35, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33641,1676076840386; forceNewPlan=false, retain=false 2023-02-11 00:54:25,237 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:25,239 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=e3beb69538b330eb0b73628e03f86d35, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,239 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676076865238"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076865238"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076865238"}]},"ts":"1676076865238"} 2023-02-11 00:54:25,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure e3beb69538b330eb0b73628e03f86d35, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:25,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e3beb69538b330eb0b73628e03f86d35, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:25,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:25,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:25,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,396 INFO [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,398 DEBUG [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1 2023-02-11 00:54:25,398 DEBUG [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1 2023-02-11 00:54:25,398 INFO [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e3beb69538b330eb0b73628e03f86d35 columnFamilyName cf1 2023-02-11 00:54:25,399 INFO [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] regionserver.HStore(310): Store=e3beb69538b330eb0b73628e03f86d35/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:25,399 INFO [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,400 DEBUG [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2 2023-02-11 00:54:25,400 DEBUG [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2 2023-02-11 00:54:25,401 INFO [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e3beb69538b330eb0b73628e03f86d35 columnFamilyName cf2 2023-02-11 00:54:25,401 INFO [StoreOpener-e3beb69538b330eb0b73628e03f86d35-1] regionserver.HStore(310): Store=e3beb69538b330eb0b73628e03f86d35/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:25,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:25,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:25,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:25,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e3beb69538b330eb0b73628e03f86d35; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74952124, jitterRate=0.11687368154525757}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:25,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e3beb69538b330eb0b73628e03f86d35: Running coprocessor pre-open hook at 1676076865395Writing region info on filesystem at 1676076865395Initializing all the Stores at 1676076865396 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076865396Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076865396Cleaning up temporary data from old regions at 1676076865403 (+7 ms)Cleaning up detritus from prior splits at 1676076865403Running coprocessor post-open hooks at 1676076865408 (+5 ms)Region opened successfully at 1676076865408 2023-02-11 00:54:25,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35., pid=71, masterSystemTime=1676076865392 2023-02-11 00:54:25,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,411 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=e3beb69538b330eb0b73628e03f86d35, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,411 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676076865411"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076865411"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076865411"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076865411"}]},"ts":"1676076865411"} 2023-02-11 00:54:25,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-11 00:54:25,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure e3beb69538b330eb0b73628e03f86d35, server=jenkins-hbase4.apache.org,33641,1676076840386 in 172 msec 2023-02-11 00:54:25,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-11 00:54:25,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e3beb69538b330eb0b73628e03f86d35, ASSIGN in 329 msec 2023-02-11 00:54:25,416 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:54:25,416 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076865416"}]},"ts":"1676076865416"} 2023-02-11 00:54:25,417 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-11 00:54:25,420 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:54:25,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 368 msec 2023-02-11 00:54:25,560 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:25,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,562 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:25,562 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,562 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:25,562 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:25,563 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,563 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,564 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,564 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,564 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,564 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,564 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,564 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,565 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,566 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:25,566 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:25,566 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,566 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:25,567 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 00:54:25,567 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,568 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,568 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,568 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,568 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,568 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,577 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/8b9b58ea7c2445d19bfe95ff62fa5911 2023-02-11 00:54:25,588 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/3632fceb68f54d9a996bbc3d741fe185 2023-02-11 00:54:25,592 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/8b9b58ea7c2445d19bfe95ff62fa5911 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911 2023-02-11 00:54:25,595 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:54:25,596 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/3632fceb68f54d9a996bbc3d741fe185 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185 2023-02-11 00:54:25,600 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:54:25,601 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 34ms, sequenceid=6, compaction requested=false 2023-02-11 00:54:25,601 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:25,601 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:25,601 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,601 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:25,601 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:25,601 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,601 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,601 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,604 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,604 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,604 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,604 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:25,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21fe6ecb[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:25,606 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:25,606 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,607 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,607 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:25,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:25,608 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,608 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,609 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:25,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:25,612 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:25,612 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:25,613 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:25,613 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,613 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:25,615 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,615 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,615 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,615 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,615 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,616 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,616 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,618 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,618 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,618 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,623 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,624 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,623 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:25,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,623 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,624 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,624 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:25,624 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,624 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:25,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,624 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,624 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,624 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,625 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,625 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,724 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:25,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:25,739 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:25,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:25,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,741 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:25,741 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,741 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:25,741 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:25,742 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,742 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,743 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,743 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,743 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,743 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,744 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,744 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:25,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,744 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,745 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,745 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,746 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,746 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,746 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,746 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:25,746 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:25,746 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,746 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,746 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:25,746 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:25,748 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,748 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,748 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,748 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,749 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,749 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,765 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/bec049088568420b8208c146c781ff57 2023-02-11 00:54:25,783 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/3c8c85c3c14f4a61a43fb0303037c270 2023-02-11 00:54:25,788 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/bec049088568420b8208c146c781ff57 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57 2023-02-11 00:54:25,792 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:54:25,793 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/3c8c85c3c14f4a61a43fb0303037c270 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270 2023-02-11 00:54:25,796 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:54:25,797 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 51ms, sequenceid=19, compaction requested=false 2023-02-11 00:54:25,797 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:25,797 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,797 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:25,797 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:25,797 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,797 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,797 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,799 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,799 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,799 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,799 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:25,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61e714ab[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:25,801 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:25,801 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,802 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,802 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:25,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,802 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,802 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,802 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:25,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,802 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:25,803 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,803 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,803 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:25,805 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,805 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,805 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,805 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,805 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,805 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,806 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,806 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,806 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,806 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:25,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:25,809 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:25,809 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:25,809 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:25,809 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,809 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:25,811 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,811 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,812 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,812 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,813 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,814 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,814 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,815 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,817 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,817 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:25,818 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:25,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,818 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,818 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,818 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,818 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,818 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,817 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,818 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:25,818 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:25,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,918 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:25,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:25,932 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:25,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:25,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,933 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:25,933 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,934 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:25,934 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:25,934 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,934 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,936 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:25,936 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,936 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,937 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:25,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,938 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:25,939 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:25,939 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:25,939 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,939 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:25,939 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:25,940 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,940 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,940 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,940 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,940 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,940 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,950 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/a9eef095d97c466a84cf5e45c21bf31d 2023-02-11 00:54:25,964 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/2841f32f231f438fb3463411b9e59543 2023-02-11 00:54:25,968 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/a9eef095d97c466a84cf5e45c21bf31d as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d 2023-02-11 00:54:25,972 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:54:25,973 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/2841f32f231f438fb3463411b9e59543 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543 2023-02-11 00:54:25,976 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:54:25,977 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 38ms, sequenceid=32, compaction requested=true 2023-02-11 00:54:25,977 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:25,977 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:25,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:25,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:25,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:25,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:25,982 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,982 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:25,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:25,984 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61e04c18[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:25,984 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:25,984 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,985 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,985 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,985 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,986 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,986 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:25,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:25,986 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:25,987 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,987 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,987 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:25,989 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,989 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:25,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:25,992 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:25,992 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:25,992 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:25,993 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:25,993 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,993 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:25,994 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,994 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:25,994 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,994 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:25,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,995 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,995 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:25,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,996 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:25,997 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:25,998 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:25,998 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:25,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:25,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,000 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,000 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,101 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:26,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:26,114 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:26,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:26,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,116 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,116 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,116 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:26,116 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:26,116 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,116 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,117 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,117 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,117 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,117 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,118 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,119 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,119 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,119 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:26,120 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:26,120 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,120 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:26,120 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:26,122 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,122 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,122 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,122 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,123 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,123 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,129 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/8b255d52cfcf4caf895d1e0d06cad976 2023-02-11 00:54:26,140 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/2a58391c4d584d1c8911b65de42bfe99 2023-02-11 00:54:26,144 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/8b255d52cfcf4caf895d1e0d06cad976 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b255d52cfcf4caf895d1e0d06cad976 2023-02-11 00:54:26,148 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b255d52cfcf4caf895d1e0d06cad976, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:54:26,148 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/2a58391c4d584d1c8911b65de42bfe99 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2a58391c4d584d1c8911b65de42bfe99 2023-02-11 00:54:26,152 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2a58391c4d584d1c8911b65de42bfe99, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:54:26,153 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 33ms, sequenceid=45, compaction requested=true 2023-02-11 00:54:26,153 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:26,153 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,155 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,155 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,155 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,155 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:26,157 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a63f9e7[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:26,157 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:26,157 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,158 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,158 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,158 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,159 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,159 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:26,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:26,161 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,161 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,161 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,162 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,162 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:26,166 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:26,166 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:26,166 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:26,166 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,166 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:26,170 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,170 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,171 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,171 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,171 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,171 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,172 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,172 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,173 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,174 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,174 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,176 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,176 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,176 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:26,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:26,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,177 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,277 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:26,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:26,291 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:26,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:26,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,293 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,293 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,293 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:26,293 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:26,293 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,293 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,294 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,294 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,294 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,294 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,295 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,295 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,296 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,296 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,296 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,296 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,296 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,296 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,296 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,296 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,296 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,296 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,297 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,297 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:26,297 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,299 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,300 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,307 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e 2023-02-11 00:54:26,317 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd 2023-02-11 00:54:26,321 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e 2023-02-11 00:54:26,325 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:54:26,326 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd 2023-02-11 00:54:26,330 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:54:26,331 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 34ms, sequenceid=58, compaction requested=true 2023-02-11 00:54:26,331 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:26,331 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,331 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:26,331 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:26,331 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,331 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,331 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,332 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,332 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,333 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,333 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:26,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8b95c56[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:26,335 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:26,335 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,336 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,336 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:26,336 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,336 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,336 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:26,336 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,336 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,337 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,337 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:26,336 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:26,338 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,338 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,338 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,339 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,339 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,339 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,340 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,340 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,340 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,340 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:26,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:26,343 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:26,343 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:26,343 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:26,343 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,343 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:26,345 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,345 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,345 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,345 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,345 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,346 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,346 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,347 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,349 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,349 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,349 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,353 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,353 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,353 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,353 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,353 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,353 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,354 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,354 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,354 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:26,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,353 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,354 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:26,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,354 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:26,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,354 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter because node does not exist (not an error) 2023-02-11 00:54:26,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,355 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,355 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,355 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,355 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,355 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,454 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:26,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:26,467 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:26,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:26,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,468 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,468 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,469 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:26,469 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:26,469 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,469 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,473 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,473 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,473 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,473 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,473 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,473 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:26,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,474 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,474 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,475 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,475 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:26,475 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,475 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:54:26,476 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,477 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,487 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/d72cfc4b8c1b402c933c598dd6fd8195 2023-02-11 00:54:26,500 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/db4ba8661809480bb72daaa4d99ae627 2023-02-11 00:54:26,503 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/d72cfc4b8c1b402c933c598dd6fd8195 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/d72cfc4b8c1b402c933c598dd6fd8195 2023-02-11 00:54:26,507 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/d72cfc4b8c1b402c933c598dd6fd8195, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:54:26,508 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/db4ba8661809480bb72daaa4d99ae627 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/db4ba8661809480bb72daaa4d99ae627 2023-02-11 00:54:26,511 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/db4ba8661809480bb72daaa4d99ae627, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:54:26,512 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 37ms, sequenceid=71, compaction requested=true 2023-02-11 00:54:26,512 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:26,512 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,512 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:26,512 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:26,512 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,512 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,512 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,515 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,515 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,515 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,515 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:26,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51a243a6[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:26,517 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:26,517 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,518 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,518 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:26,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,518 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:26,518 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,518 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,518 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,519 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:26,519 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,519 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:26,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,520 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:26,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:26,523 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:26,523 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:26,523 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:26,523 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,523 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:26,525 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,525 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,525 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,526 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,526 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,528 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,528 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,528 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,532 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,532 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,532 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:26,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,533 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,633 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:26,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:26,646 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:26,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:26,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,648 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,648 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,648 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:26,648 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:26,648 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,648 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,649 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,649 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,649 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,649 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,650 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,650 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,652 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,652 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,651 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,652 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:26,652 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:26,652 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,652 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:26,652 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-11 00:54:26,653 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,653 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,654 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,654 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,654 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,654 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,661 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/a7e09760cade4a90997ca6ef1b1c6178 2023-02-11 00:54:26,665 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7e09760cade4a90997ca6ef1b1c6178 2023-02-11 00:54:26,673 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/a7eb3d0116754d24be2a9080e88bfd14 2023-02-11 00:54:26,676 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7eb3d0116754d24be2a9080e88bfd14 2023-02-11 00:54:26,676 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/a7e09760cade4a90997ca6ef1b1c6178 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178 2023-02-11 00:54:26,680 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7e09760cade4a90997ca6ef1b1c6178 2023-02-11 00:54:26,680 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178, entries=9, sequenceid=83, filesize=5.1 K 2023-02-11 00:54:26,681 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/a7eb3d0116754d24be2a9080e88bfd14 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14 2023-02-11 00:54:26,684 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7eb3d0116754d24be2a9080e88bfd14 2023-02-11 00:54:26,684 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14, entries=9, sequenceid=83, filesize=5.1 K 2023-02-11 00:54:26,685 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 33ms, sequenceid=83, compaction requested=true 2023-02-11 00:54:26,685 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:26,685 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,685 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:26,685 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:26,685 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,685 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,685 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,687 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,687 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,687 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,687 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:26,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32c15506[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:26,690 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:26,690 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:26,692 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,692 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:26,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,692 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:26,693 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,693 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,694 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,694 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,697 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,697 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,697 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:26,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:26,701 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:26,701 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:26,701 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:26,701 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,701 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:26,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,704 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,705 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,706 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,706 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,709 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,709 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,710 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:26,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,709 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,810 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:26,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:26,819 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:26,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:26,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,820 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,820 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,820 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:26,820 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:26,821 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,821 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,823 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,823 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,823 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,823 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,823 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,823 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:26,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,824 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 00:54:26,824 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,824 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,824 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,824 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,824 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,824 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:26,825 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:26,825 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,825 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:26,825 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-11 00:54:26,826 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,827 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,827 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,827 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,827 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,827 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,835 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:26,839 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:26,847 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:26,851 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:26,851 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/cae7d5d1979e44e8a4ffa24492c5599a as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:26,855 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:26,855 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/cae7d5d1979e44e8a4ffa24492c5599a, entries=5, sequenceid=91, filesize=5.0 K 2023-02-11 00:54:26,856 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/9d0630de832e4c57aeac9c6a951a1af4 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:26,859 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:26,859 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/9d0630de832e4c57aeac9c6a951a1af4, entries=5, sequenceid=91, filesize=5.0 K 2023-02-11 00:54:26,859 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 34ms, sequenceid=91, compaction requested=true 2023-02-11 00:54:26,860 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:26,860 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,860 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:26,860 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:26,860 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,860 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,860 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,861 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,861 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,861 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,862 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:26,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77a83e78[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:26,863 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:26,863 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,865 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,865 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,865 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,865 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:26,866 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:26,866 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:26,867 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,867 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,867 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,867 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,867 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,867 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:26,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,867 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,867 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,868 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,868 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:26,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:26,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:26,870 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:26,870 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:26,870 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:26,870 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,870 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:26,871 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,871 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:26,871 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,871 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,871 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:26,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,872 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,872 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:26,873 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:26,874 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,874 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,875 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,880 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,880 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,880 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,880 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,881 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,880 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,881 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,881 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:26,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:26,881 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,881 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:26,881 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,881 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:26,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,881 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:26,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:26,881 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,882 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,882 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,882 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:26,882 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:26,882 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,904 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,982 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:26,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:26,990 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:54:26,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:54:26,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,991 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:26,991 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,991 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 00:54:26,991 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 00:54:26,992 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,992 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,993 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,993 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,993 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,993 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:26,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:26,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:26,993 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:26,993 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:54:26,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,995 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,995 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 00:54:26,995 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:54:26,995 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. started... 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:26,995 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:26,996 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-11 00:54:26,996 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,997 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,997 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,997 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:26,998 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:26,998 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:27,005 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:27,009 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:27,017 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:27,021 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:27,021 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/b17ed9f4b97c41bfb77130869482f7dd as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:27,025 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:27,025 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/b17ed9f4b97c41bfb77130869482f7dd, entries=5, sequenceid=99, filesize=5.0 K 2023-02-11 00:54:27,026 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/f06fe317287248ed98874d1821f6e188 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:27,029 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:27,029 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/f06fe317287248ed98874d1821f6e188, entries=5, sequenceid=99, filesize=5.0 K 2023-02-11 00:54:27,030 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 34ms, sequenceid=99, compaction requested=true 2023-02-11 00:54:27,030 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e3beb69538b330eb0b73628e03f86d35: 2023-02-11 00:54:27,030 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:27,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:54:27,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:54:27,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 00:54:27,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 00:54:27,034 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,034 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:27,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:27,034 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,034 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:54:27,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:27,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:27,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:27,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 00:54:27,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35b19737[Count = 0] remaining members to acquire global barrier 2023-02-11 00:54:27,036 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 00:54:27,036 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:27,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:27,037 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,037 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:54:27,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,037 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:54:27,038 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:27,038 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:27,038 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:54:27,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,038 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 00:54:27,038 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 00:54:27,038 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:27,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:27,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:27,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:27,039 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 00:54:27,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:27,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:27,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:27,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:54:27,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:54:27,042 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 00:54:27,042 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:54:27,042 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:54:27,042 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,042 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:54:27,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:27,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:54:27,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:54:27,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:27,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:27,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:27,043 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:27,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:54:27,044 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:27,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:27,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:27,044 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:27,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,045 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:54:27,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:54:27,046 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,046 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,046 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,049 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,049 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 00:54:27,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:54:27,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:27,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:27,049 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:27,050 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,050 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:54:27,050 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:54:27,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:54:27,050 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:54:27,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:27,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:27,050 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:27,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:54:27,050 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:54:27,050 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:27,051 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:27,051 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,051 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:27,150 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:54:27,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:54:27,153 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,154 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:27,155 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:27,155 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:27,155 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:27,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e3beb69538b330eb0b73628e03f86d35, UNASSIGN}] 2023-02-11 00:54:27,157 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e3beb69538b330eb0b73628e03f86d35, UNASSIGN 2023-02-11 00:54:27,158 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=e3beb69538b330eb0b73628e03f86d35, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,158 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676076867158"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076867158"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076867158"}]},"ts":"1676076867158"} 2023-02-11 00:54:27,159 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure e3beb69538b330eb0b73628e03f86d35, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:27,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e3beb69538b330eb0b73628e03f86d35, disabling compactions & flushes 2023-02-11 00:54:27,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:27,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:27,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. after waiting 0 ms 2023-02-11 00:54:27,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:27,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing e3beb69538b330eb0b73628e03f86d35 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-11 00:54:27,321 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:27,325 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:27,332 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:27,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:27,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf1/32ffc6b68838469e945f38e9619a0a25 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:27,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:27,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/32ffc6b68838469e945f38e9619a0a25, entries=1, sequenceid=103, filesize=4.9 K 2023-02-11 00:54:27,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.tmp/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:27,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:27,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e, entries=1, sequenceid=103, filesize=4.9 K 2023-02-11 00:54:27,343 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 32ms, sequenceid=103, compaction requested=true 2023-02-11 00:54:27,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-11 00:54:27,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:27,348 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. 2023-02-11 00:54:27,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e3beb69538b330eb0b73628e03f86d35: Waiting for close lock at 1676076867311Running coprocessor pre-close hooks at 1676076867311Disabling compacts and flushes for region at 1676076867311Disabling writes for close at 1676076867311Obtaining lock to block concurrent updates at 1676076867311Preparing flush snapshotting stores in e3beb69538b330eb0b73628e03f86d35 at 1676076867311Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676076867312 (+1 ms)Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35. at 1676076867312Flushing e3beb69538b330eb0b73628e03f86d35/cf1: creating writer at 1676076867313 (+1 ms)Flushing e3beb69538b330eb0b73628e03f86d35/cf1: appending metadata at 1676076867315 (+2 ms)Flushing e3beb69538b330eb0b73628e03f86d35/cf1: closing flushed file at 1676076867315Flushing e3beb69538b330eb0b73628e03f86d35/cf2: creating writer at 1676076867325 (+10 ms)Flushing e3beb69538b330eb0b73628e03f86d35/cf2: appending metadata at 1676076867326 (+1 ms)Flushing e3beb69538b330eb0b73628e03f86d35/cf2: closing flushed file at 1676076867326Flushing e3beb69538b330eb0b73628e03f86d35/cf1: reopening flushed file at 1676076867336 (+10 ms)Flushing e3beb69538b330eb0b73628e03f86d35/cf2: reopening flushed file at 1676076867340 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for e3beb69538b330eb0b73628e03f86d35 in 32ms, sequenceid=103, compaction requested=true at 1676076867343 (+3 ms)Writing region close event to WAL at 1676076867345 (+2 ms)Running coprocessor post-close hooks at 1676076867348 (+3 ms)Closed at 1676076867348 2023-02-11 00:54:27,350 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,350 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=e3beb69538b330eb0b73628e03f86d35, regionState=CLOSED 2023-02-11 00:54:27,350 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676076867350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076867350"}]},"ts":"1676076867350"} 2023-02-11 00:54:27,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-11 00:54:27,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure e3beb69538b330eb0b73628e03f86d35, server=jenkins-hbase4.apache.org,33641,1676076840386 in 192 msec 2023-02-11 00:54:27,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-11 00:54:27,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e3beb69538b330eb0b73628e03f86d35, UNASSIGN in 196 msec 2023-02-11 00:54:27,369 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=e3beb69538b330eb0b73628e03f86d35, threads=16 2023-02-11 00:54:27,369 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,369 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2a58391c4d584d1c8911b65de42bfe99 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,370 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,372 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,374 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,376 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/9d0630de832e4c57aeac9c6a951a1af4 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,376 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,380 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,381 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/db4ba8661809480bb72daaa4d99ae627 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,381 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/f06fe317287248ed98874d1821f6e188 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,388 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,388 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/32ffc6b68838469e945f38e9619a0a25 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,389 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b255d52cfcf4caf895d1e0d06cad976 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,389 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,392 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,393 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,395 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:27,397 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2a58391c4d584d1c8911b65de42bfe99, top=false 2023-02-11 00:54:27,404 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7e09760cade4a90997ca6ef1b1c6178 2023-02-11 00:54:27,405 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:27,406 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/db4ba8661809480bb72daaa4d99ae627, top=false 2023-02-11 00:54:27,408 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7eb3d0116754d24be2a9080e88bfd14 2023-02-11 00:54:27,408 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd, top=false 2023-02-11 00:54:27,410 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b255d52cfcf4caf895d1e0d06cad976, top=false 2023-02-11 00:54:27,424 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:27,425 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e, top=false 2023-02-11 00:54:27,427 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:27,427 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:27,427 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/9d0630de832e4c57aeac9c6a951a1af4, top=true 2023-02-11 00:54:27,433 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:27,433 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/f06fe317287248ed98874d1821f6e188, top=true 2023-02-11 00:54:27,434 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:27,435 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e, top=true 2023-02-11 00:54:27,435 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:27,435 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/32ffc6b68838469e945f38e9619a0a25, top=true 2023-02-11 00:54:27,453 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-2a58391c4d584d1c8911b65de42bfe99 for child: 3672dd666d16ab8bcdec33d37c2608f2, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,474 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2a58391c4d584d1c8911b65de42bfe99 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,475 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/b17ed9f4b97c41bfb77130869482f7dd for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,478 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:27,490 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-9d0630de832e4c57aeac9c6a951a1af4 for child: 6f03a7215e0b89c529706aaf04e65832, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,490 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/9d0630de832e4c57aeac9c6a951a1af4 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,490 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,492 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-0dd43a93d5d04d2cb68aa4692d9cb92e for child: 3672dd666d16ab8bcdec33d37c2608f2, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,496 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-db4ba8661809480bb72daaa4d99ae627 for child: 3672dd666d16ab8bcdec33d37c2608f2, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,498 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:27,498 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/b17ed9f4b97c41bfb77130869482f7dd, top=true 2023-02-11 00:54:27,502 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/db4ba8661809480bb72daaa4d99ae627 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,502 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/cae7d5d1979e44e8a4ffa24492c5599a for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,502 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,502 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/d72cfc4b8c1b402c933c598dd6fd8195 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,505 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:27,505 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/d72cfc4b8c1b402c933c598dd6fd8195, top=false 2023-02-11 00:54:27,508 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:27,508 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/cae7d5d1979e44e8a4ffa24492c5599a, top=true 2023-02-11 00:54:27,516 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-4eba672f17fe4f24bfb2e7a9cfc147fd for child: 3672dd666d16ab8bcdec33d37c2608f2, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,516 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-8b255d52cfcf4caf895d1e0d06cad976 for child: 3672dd666d16ab8bcdec33d37c2608f2, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,535 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,536 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b255d52cfcf4caf895d1e0d06cad976 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,541 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-32ffc6b68838469e945f38e9619a0a25 for child: 6f03a7215e0b89c529706aaf04e65832, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,541 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/32ffc6b68838469e945f38e9619a0a25 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,541 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-c1e64bae4e3a4671bcdd87256b5a0e2e for child: 6f03a7215e0b89c529706aaf04e65832, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,541 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,541 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-f06fe317287248ed98874d1821f6e188 for child: 6f03a7215e0b89c529706aaf04e65832, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,541 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/f06fe317287248ed98874d1821f6e188 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,552 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7eb3d0116754d24be2a9080e88bfd14 2023-02-11 00:54:27,553 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7e09760cade4a90997ca6ef1b1c6178 2023-02-11 00:54:27,573 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-cae7d5d1979e44e8a4ffa24492c5599a for child: 6f03a7215e0b89c529706aaf04e65832, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,573 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/cae7d5d1979e44e8a4ffa24492c5599a for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,577 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-b17ed9f4b97c41bfb77130869482f7dd for child: 6f03a7215e0b89c529706aaf04e65832, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,577 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/b17ed9f4b97c41bfb77130869482f7dd for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,588 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,591 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,591 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/.splits/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-d72cfc4b8c1b402c933c598dd6fd8195 for child: 3672dd666d16ab8bcdec33d37c2608f2, parent: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,599 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/d72cfc4b8c1b402c933c598dd6fd8195 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,616 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,616 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,617 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,619 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,623 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,630 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57 for region: e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:27,630 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region e3beb69538b330eb0b73628e03f86d35 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:54:27,663 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-11 00:54:27,665 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-11 00:54:27,667 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676076867667"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676076867667"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676076867667"}]},"ts":"1676076867667"} 2023-02-11 00:54:27,667 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076867667"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076867667"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076867667"}]},"ts":"1676076867667"} 2023-02-11 00:54:27,667 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076867667"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076867667"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076867667"}]},"ts":"1676076867667"} 2023-02-11 00:54:27,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=3672dd666d16ab8bcdec33d37c2608f2, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6f03a7215e0b89c529706aaf04e65832, ASSIGN}] 2023-02-11 00:54:27,670 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6f03a7215e0b89c529706aaf04e65832, ASSIGN 2023-02-11 00:54:27,671 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6f03a7215e0b89c529706aaf04e65832, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33641,1676076840386; forceNewPlan=false, retain=false 2023-02-11 00:54:27,671 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=3672dd666d16ab8bcdec33d37c2608f2, ASSIGN 2023-02-11 00:54:27,672 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=3672dd666d16ab8bcdec33d37c2608f2, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33641,1676076840386; forceNewPlan=false, retain=false 2023-02-11 00:54:27,821 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 00:54:27,823 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=3672dd666d16ab8bcdec33d37c2608f2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,823 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076867823"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076867823"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076867823"}]},"ts":"1676076867823"} 2023-02-11 00:54:27,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 3672dd666d16ab8bcdec33d37c2608f2, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:27,826 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=6f03a7215e0b89c529706aaf04e65832, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:27,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076867826"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076867826"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076867826"}]},"ts":"1676076867826"} 2023-02-11 00:54:27,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 6f03a7215e0b89c529706aaf04e65832, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:27,944 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 00:54:27,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:27,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6f03a7215e0b89c529706aaf04e65832, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.', STARTKEY => '60', ENDKEY => ''} 2023-02-11 00:54:27,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:27,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:27,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:27,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:27,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:27,981 INFO [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:27,981 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1 2023-02-11 00:54:27,982 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1 2023-02-11 00:54:27,982 INFO [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f03a7215e0b89c529706aaf04e65832 columnFamilyName cf1 2023-02-11 00:54:27,996 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911-top 2023-02-11 00:54:27,999 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,000 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178-top 2023-02-11 00:54:28,004 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d-top 2023-02-11 00:54:28,007 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57-top 2023-02-11 00:54:28,010 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:28,010 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:28,013 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:28,013 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:28,016 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:28,016 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:28,016 INFO [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(310): Store=6f03a7215e0b89c529706aaf04e65832/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:28,016 INFO [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:28,017 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2 2023-02-11 00:54:28,017 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2 2023-02-11 00:54:28,018 INFO [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f03a7215e0b89c529706aaf04e65832 columnFamilyName cf2 2023-02-11 00:54:28,028 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543-top 2023-02-11 00:54:28,033 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185-top 2023-02-11 00:54:28,037 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270-top 2023-02-11 00:54:28,040 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,041 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14-top 2023-02-11 00:54:28,044 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:28,044 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:28,047 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:28,047 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:28,050 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:28,050 DEBUG [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:28,051 INFO [StoreOpener-6f03a7215e0b89c529706aaf04e65832-1] regionserver.HStore(310): Store=6f03a7215e0b89c529706aaf04e65832/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:28,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:28,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:28,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:28,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:28,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6f03a7215e0b89c529706aaf04e65832; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68767013, jitterRate=0.024708345532417297}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:28,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6f03a7215e0b89c529706aaf04e65832: Running coprocessor pre-open hook at 1676076867979Writing region info on filesystem at 1676076867979Initializing all the Stores at 1676076867980 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076867980Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076867980Cleaning up temporary data from old regions at 1676076868053 (+73 ms)Cleaning up detritus from prior splits at 1676076868053Running coprocessor post-open hooks at 1676076868056 (+3 ms)Region opened successfully at 1676076868056 2023-02-11 00:54:28,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832., pid=78, masterSystemTime=1676076867975 2023-02-11 00:54:28,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,058 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:28,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:54:28,058 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:28,058 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,058 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1912): 6f03a7215e0b89c529706aaf04e65832/cf1 is initiating minor compaction (all files) 2023-02-11 00:54:28,058 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,058 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6f03a7215e0b89c529706aaf04e65832/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,058 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1912): 6f03a7215e0b89c529706aaf04e65832/cf2 is initiating minor compaction (all files) 2023-02-11 00:54:28,058 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6f03a7215e0b89c529706aaf04e65832/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,058 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-cae7d5d1979e44e8a4ffa24492c5599a, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-b17ed9f4b97c41bfb77130869482f7dd, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-32ffc6b68838469e945f38e9619a0a25] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/.tmp, totalSize=34.8 K 2023-02-11 00:54:28,058 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-9d0630de832e4c57aeac9c6a951a1af4, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-f06fe317287248ed98874d1821f6e188, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-c1e64bae4e3a4671bcdd87256b5a0e2e] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/.tmp, totalSize=34.8 K 2023-02-11 00:54:28,059 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076865555 2023-02-11 00:54:28,059 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076865555 2023-02-11 00:54:28,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,059 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076865726 2023-02-11 00:54:28,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3672dd666d16ab8bcdec33d37c2608f2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.', STARTKEY => '', ENDKEY => '60'} 2023-02-11 00:54:28,059 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076865726 2023-02-11 00:54:28,060 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076865920 2023-02-11 00:54:28,060 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=6f03a7215e0b89c529706aaf04e65832, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:28,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:28,060 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076865920 2023-02-11 00:54:28,060 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076868060"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076868060"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076868060"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076868060"}]},"ts":"1676076868060"} 2023-02-11 00:54:28,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:28,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,060 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676076866634 2023-02-11 00:54:28,061 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676076866634 2023-02-11 00:54:28,061 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-9d0630de832e4c57aeac9c6a951a1af4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-11 00:54:28,061 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-cae7d5d1979e44e8a4ffa24492c5599a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-11 00:54:28,062 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-b17ed9f4b97c41bfb77130869482f7dd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-11 00:54:28,062 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-f06fe317287248ed98874d1821f6e188, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-11 00:54:28,062 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-32ffc6b68838469e945f38e9619a0a25, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 00:54:28,062 INFO [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,063 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-c1e64bae4e3a4671bcdd87256b5a0e2e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 00:54:28,064 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1 2023-02-11 00:54:28,064 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1 2023-02-11 00:54:28,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-11 00:54:28,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 6f03a7215e0b89c529706aaf04e65832, server=jenkins-hbase4.apache.org,33641,1676076840386 in 235 msec 2023-02-11 00:54:28,065 INFO [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3672dd666d16ab8bcdec33d37c2608f2 columnFamilyName cf1 2023-02-11 00:54:28,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6f03a7215e0b89c529706aaf04e65832, ASSIGN in 395 msec 2023-02-11 00:54:28,082 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] throttle.PressureAwareThroughputController(145): 6f03a7215e0b89c529706aaf04e65832#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:28,089 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911-bottom 2023-02-11 00:54:28,093 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,093 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178-bottom 2023-02-11 00:54:28,095 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6f03a7215e0b89c529706aaf04e65832#cf1#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:28,097 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d-bottom 2023-02-11 00:54:28,100 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57-bottom 2023-02-11 00:54:28,103 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-0dd43a93d5d04d2cb68aa4692d9cb92e 2023-02-11 00:54:28,108 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-8b255d52cfcf4caf895d1e0d06cad976 2023-02-11 00:54:28,110 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for b4c95f93e29944a5bc8405dcc5215b6b: NONE, but ROW specified in column family configuration 2023-02-11 00:54:28,110 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/.tmp/cf2/b4c95f93e29944a5bc8405dcc5215b6b as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/b4c95f93e29944a5bc8405dcc5215b6b 2023-02-11 00:54:28,111 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 17fb51d03fd746b79bf4f5f6bf303760: NONE, but ROW specified in column family configuration 2023-02-11 00:54:28,112 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/.tmp/cf1/17fb51d03fd746b79bf4f5f6bf303760 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/17fb51d03fd746b79bf4f5f6bf303760 2023-02-11 00:54:28,112 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-d72cfc4b8c1b402c933c598dd6fd8195 2023-02-11 00:54:28,112 INFO [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(310): Store=3672dd666d16ab8bcdec33d37c2608f2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:28,112 INFO [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,113 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2 2023-02-11 00:54:28,113 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2 2023-02-11 00:54:28,114 INFO [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3672dd666d16ab8bcdec33d37c2608f2 columnFamilyName cf2 2023-02-11 00:54:28,115 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for b4c95f93e29944a5bc8405dcc5215b6b: NONE, but ROW specified in column family configuration 2023-02-11 00:54:28,116 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6f03a7215e0b89c529706aaf04e65832/cf2 of 6f03a7215e0b89c529706aaf04e65832 into b4c95f93e29944a5bc8405dcc5215b6b(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:28,116 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6f03a7215e0b89c529706aaf04e65832: 2023-02-11 00:54:28,116 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832., storeName=6f03a7215e0b89c529706aaf04e65832/cf2, priority=9, startTime=1676076868058; duration=0sec 2023-02-11 00:54:28,116 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,116 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 17fb51d03fd746b79bf4f5f6bf303760: NONE, but ROW specified in column family configuration 2023-02-11 00:54:28,117 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6f03a7215e0b89c529706aaf04e65832/cf1 of 6f03a7215e0b89c529706aaf04e65832 into 17fb51d03fd746b79bf4f5f6bf303760(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:28,117 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6f03a7215e0b89c529706aaf04e65832: 2023-02-11 00:54:28,117 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832., storeName=6f03a7215e0b89c529706aaf04e65832/cf1, priority=9, startTime=1676076868056; duration=0sec 2023-02-11 00:54:28,117 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,122 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543-bottom 2023-02-11 00:54:28,125 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185-bottom 2023-02-11 00:54:28,129 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270-bottom 2023-02-11 00:54:28,131 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,132 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14-bottom 2023-02-11 00:54:28,134 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-2a58391c4d584d1c8911b65de42bfe99 2023-02-11 00:54:28,136 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-4eba672f17fe4f24bfb2e7a9cfc147fd 2023-02-11 00:54:28,140 DEBUG [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-db4ba8661809480bb72daaa4d99ae627 2023-02-11 00:54:28,140 INFO [StoreOpener-3672dd666d16ab8bcdec33d37c2608f2-1] regionserver.HStore(310): Store=3672dd666d16ab8bcdec33d37c2608f2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:28,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:54:28,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3672dd666d16ab8bcdec33d37c2608f2; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67911871, jitterRate=0.011965736746788025}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:54:28,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3672dd666d16ab8bcdec33d37c2608f2: Running coprocessor pre-open hook at 1676076868061Writing region info on filesystem at 1676076868061Initializing all the Stores at 1676076868062 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076868062Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076868062Cleaning up temporary data from old regions at 1676076868142 (+80 ms)Cleaning up detritus from prior splits at 1676076868142Running coprocessor post-open hooks at 1676076868144 (+2 ms)Region opened successfully at 1676076868144 2023-02-11 00:54:28,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2., pid=77, masterSystemTime=1676076867975 2023-02-11 00:54:28,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,145 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:28,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:54:28,146 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:54:28,146 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,146 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1912): 3672dd666d16ab8bcdec33d37c2608f2/cf1 is initiating minor compaction (all files) 2023-02-11 00:54:28,146 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,146 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1912): 3672dd666d16ab8bcdec33d37c2608f2/cf2 is initiating minor compaction (all files) 2023-02-11 00:54:28,146 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3672dd666d16ab8bcdec33d37c2608f2/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,146 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3672dd666d16ab8bcdec33d37c2608f2/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,146 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-2a58391c4d584d1c8911b65de42bfe99, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-4eba672f17fe4f24bfb2e7a9cfc147fd, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-db4ba8661809480bb72daaa4d99ae627, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/.tmp, totalSize=35.0 K 2023-02-11 00:54:28,146 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-8b255d52cfcf4caf895d1e0d06cad976, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-0dd43a93d5d04d2cb68aa4692d9cb92e, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-d72cfc4b8c1b402c933c598dd6fd8195, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/.tmp, totalSize=35.0 K 2023-02-11 00:54:28,147 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076865555 2023-02-11 00:54:28,147 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076865555 2023-02-11 00:54:28,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,147 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076865726 2023-02-11 00:54:28,147 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076865726 2023-02-11 00:54:28,148 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=3672dd666d16ab8bcdec33d37c2608f2, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:28,148 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076865920 2023-02-11 00:54:28,148 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076865920 2023-02-11 00:54:28,148 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076868148"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076868148"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076868148"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076868148"}]},"ts":"1676076868148"} 2023-02-11 00:54:28,148 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-2a58391c4d584d1c8911b65de42bfe99, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076866103 2023-02-11 00:54:28,148 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-8b255d52cfcf4caf895d1e0d06cad976, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076866103 2023-02-11 00:54:28,148 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-4eba672f17fe4f24bfb2e7a9cfc147fd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076866279 2023-02-11 00:54:28,149 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-0dd43a93d5d04d2cb68aa4692d9cb92e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076866279 2023-02-11 00:54:28,149 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-db4ba8661809480bb72daaa4d99ae627, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076866456 2023-02-11 00:54:28,149 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-d72cfc4b8c1b402c933c598dd6fd8195, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076866456 2023-02-11 00:54:28,149 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076866634 2023-02-11 00:54:28,150 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076866634 2023-02-11 00:54:28,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-11 00:54:28,152 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 3672dd666d16ab8bcdec33d37c2608f2, server=jenkins-hbase4.apache.org,33641,1676076840386 in 325 msec 2023-02-11 00:54:28,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-11 00:54:28,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=3672dd666d16ab8bcdec33d37c2608f2, ASSIGN in 482 msec 2023-02-11 00:54:28,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 in 1.0020 sec 2023-02-11 00:54:28,165 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3672dd666d16ab8bcdec33d37c2608f2#cf1#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:28,166 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] throttle.PressureAwareThroughputController(145): 3672dd666d16ab8bcdec33d37c2608f2#cf2#compaction#75 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:28,188 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/.tmp/cf2/2364f56374aa4a7f8860d49550f4a66c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2364f56374aa4a7f8860d49550f4a66c 2023-02-11 00:54:28,189 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/.tmp/cf1/87e4ec8aa9fd49d9a108e5ad327526d5 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/87e4ec8aa9fd49d9a108e5ad327526d5 2023-02-11 00:54:28,193 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3672dd666d16ab8bcdec33d37c2608f2/cf2 of 3672dd666d16ab8bcdec33d37c2608f2 into 2364f56374aa4a7f8860d49550f4a66c(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:28,193 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3672dd666d16ab8bcdec33d37c2608f2: 2023-02-11 00:54:28,193 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2., storeName=3672dd666d16ab8bcdec33d37c2608f2/cf2, priority=9, startTime=1676076868145; duration=0sec 2023-02-11 00:54:28,193 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,194 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3672dd666d16ab8bcdec33d37c2608f2/cf1 of 3672dd666d16ab8bcdec33d37c2608f2 into 87e4ec8aa9fd49d9a108e5ad327526d5(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:28,194 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3672dd666d16ab8bcdec33d37c2608f2: 2023-02-11 00:54:28,194 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2., storeName=3672dd666d16ab8bcdec33d37c2608f2/cf1, priority=9, startTime=1676076868144; duration=0sec 2023-02-11 00:54:28,195 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,415 DEBUG [Listener at localhost/35701] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 00:54:28,419 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54776, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 00:54:28,420 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,420 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 00:54:28,421 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] regionserver.HStore(1912): 3672dd666d16ab8bcdec33d37c2608f2/cf1 is initiating major compaction (all files) 2023-02-11 00:54:28,421 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7b0040e2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,421 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3672dd666d16ab8bcdec33d37c2608f2/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,421 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 00:54:28,421 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/87e4ec8aa9fd49d9a108e5ad327526d5] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/.tmp, totalSize=6.8 K 2023-02-11 00:54:28,421 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] regionserver.HStore(1912): 3672dd666d16ab8bcdec33d37c2608f2/cf2 is initiating major compaction (all files) 2023-02-11 00:54:28,421 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2e718ef2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,421 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3672dd666d16ab8bcdec33d37c2608f2/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,421 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2364f56374aa4a7f8860d49550f4a66c] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/.tmp, totalSize=6.8 K 2023-02-11 00:54:28,421 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 87e4ec8aa9fd49d9a108e5ad327526d5, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076865555 2023-02-11 00:54:28,422 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 2364f56374aa4a7f8860d49550f4a66c, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076865555 2023-02-11 00:54:28,423 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,423 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 00:54:28,423 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] regionserver.HStore(1912): 6f03a7215e0b89c529706aaf04e65832/cf1 is initiating major compaction (all files) 2023-02-11 00:54:28,423 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3bd1f7c4; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:54:28,423 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 00:54:28,423 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] regionserver.HStore(1912): 6f03a7215e0b89c529706aaf04e65832/cf2 is initiating major compaction (all files) 2023-02-11 00:54:28,423 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@731558e6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 00:54:28,424 INFO [Listener at localhost/35701] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-11 00:54:28,426 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] throttle.PressureAwareThroughputController(145): 3672dd666d16ab8bcdec33d37c2608f2#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:28,426 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3672dd666d16ab8bcdec33d37c2608f2#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:28,439 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. is not mergeable because it has references 2023-02-11 00:54:28,440 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. is not mergeable because it has references 2023-02-11 00:54:28,441 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/.tmp/cf2/429b389ca60b428b92e13f7a0da5a38c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/429b389ca60b428b92e13f7a0da5a38c 2023-02-11 00:54:28,443 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/.tmp/cf1/a7f6e0c983dd4ef199cabf0960ea3e7c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a7f6e0c983dd4ef199cabf0960ea3e7c 2023-02-11 00:54:28,446 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 3672dd666d16ab8bcdec33d37c2608f2/cf2 of 3672dd666d16ab8bcdec33d37c2608f2 into 429b389ca60b428b92e13f7a0da5a38c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:28,446 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3672dd666d16ab8bcdec33d37c2608f2: 2023-02-11 00:54:28,447 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 3672dd666d16ab8bcdec33d37c2608f2/cf1 of 3672dd666d16ab8bcdec33d37c2608f2 into a7f6e0c983dd4ef199cabf0960ea3e7c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:28,447 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676076868421; duration=0sec 2023-02-11 00:54:28,447 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3672dd666d16ab8bcdec33d37c2608f2: 2023-02-11 00:54:28,447 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676076868421; duration=0sec 2023-02-11 00:54:28,447 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 00:54:28,448 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:54:28,448 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6f03a7215e0b89c529706aaf04e65832/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,448 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6f03a7215e0b89c529706aaf04e65832/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,448 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/17fb51d03fd746b79bf4f5f6bf303760] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/.tmp, totalSize=5.0 K 2023-02-11 00:54:28,448 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/b4c95f93e29944a5bc8405dcc5215b6b] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/.tmp, totalSize=5.0 K 2023-02-11 00:54:28,448 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 17fb51d03fd746b79bf4f5f6bf303760, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 00:54:28,448 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting b4c95f93e29944a5bc8405dcc5215b6b, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 00:54:28,450 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] throttle.PressureAwareThroughputController(145): 6f03a7215e0b89c529706aaf04e65832#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:28,451 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6f03a7215e0b89c529706aaf04e65832#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:54:28,464 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6d7d2be88cbf45f3b9f765e55d2bc6a0: NONE, but ROW specified in column family configuration 2023-02-11 00:54:28,465 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/.tmp/cf1/6d7d2be88cbf45f3b9f765e55d2bc6a0 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/6d7d2be88cbf45f3b9f765e55d2bc6a0 2023-02-11 00:54:28,468 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0bb9f1c9b5b24fe8b4318a63491cb733: NONE, but ROW specified in column family configuration 2023-02-11 00:54:28,468 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/.tmp/cf2/0bb9f1c9b5b24fe8b4318a63491cb733 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/0bb9f1c9b5b24fe8b4318a63491cb733 2023-02-11 00:54:28,468 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6d7d2be88cbf45f3b9f765e55d2bc6a0: NONE, but ROW specified in column family configuration 2023-02-11 00:54:28,469 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 6f03a7215e0b89c529706aaf04e65832/cf1 of 6f03a7215e0b89c529706aaf04e65832 into 6d7d2be88cbf45f3b9f765e55d2bc6a0(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:28,469 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6f03a7215e0b89c529706aaf04e65832: 2023-02-11 00:54:28,470 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676076868423; duration=0sec 2023-02-11 00:54:28,470 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,472 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0bb9f1c9b5b24fe8b4318a63491cb733: NONE, but ROW specified in column family configuration 2023-02-11 00:54:28,472 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 6f03a7215e0b89c529706aaf04e65832/cf2 of 6f03a7215e0b89c529706aaf04e65832 into 0bb9f1c9b5b24fe8b4318a63491cb733(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:54:28,473 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6f03a7215e0b89c529706aaf04e65832: 2023-02-11 00:54:28,473 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676076868423; duration=0sec 2023-02-11 00:54:28,473 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:54:28,547 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33641] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. is not mergeable because it has references 2023-02-11 00:54:28,548 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33641] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. is not mergeable because it has references 2023-02-11 00:54:28,553 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-11 00:54:28,555 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:28,556 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,557 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 00:54:28,560 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076868560"}]},"ts":"1676076868560"} 2023-02-11 00:54:28,561 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-11 00:54:28,563 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-11 00:54:28,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=3672dd666d16ab8bcdec33d37c2608f2, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6f03a7215e0b89c529706aaf04e65832, UNASSIGN}] 2023-02-11 00:54:28,564 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6f03a7215e0b89c529706aaf04e65832, UNASSIGN 2023-02-11 00:54:28,564 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=6f03a7215e0b89c529706aaf04e65832, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:28,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076868564"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076868564"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076868564"}]},"ts":"1676076868564"} 2023-02-11 00:54:28,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 6f03a7215e0b89c529706aaf04e65832, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:28,567 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=3672dd666d16ab8bcdec33d37c2608f2, UNASSIGN 2023-02-11 00:54:28,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=3672dd666d16ab8bcdec33d37c2608f2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:54:28,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076868568"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076868568"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076868568"}]},"ts":"1676076868568"} 2023-02-11 00:54:28,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 3672dd666d16ab8bcdec33d37c2608f2, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:54:28,660 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 00:54:28,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3672dd666d16ab8bcdec33d37c2608f2, disabling compactions & flushes 2023-02-11 00:54:28,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. after waiting 0 ms 2023-02-11 00:54:28,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,718 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-8b255d52cfcf4caf895d1e0d06cad976, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-0dd43a93d5d04d2cb68aa4692d9cb92e, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-d72cfc4b8c1b402c933c598dd6fd8195, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/87e4ec8aa9fd49d9a108e5ad327526d5, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178-bottom] to archive 2023-02-11 00:54:28,719 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:28,720 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,721 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,722 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,723 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-8b255d52cfcf4caf895d1e0d06cad976 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-8b255d52cfcf4caf895d1e0d06cad976 2023-02-11 00:54:28,724 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-0dd43a93d5d04d2cb68aa4692d9cb92e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-0dd43a93d5d04d2cb68aa4692d9cb92e 2023-02-11 00:54:28,725 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-d72cfc4b8c1b402c933c598dd6fd8195 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-d72cfc4b8c1b402c933c598dd6fd8195 2023-02-11 00:54:28,726 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/87e4ec8aa9fd49d9a108e5ad327526d5 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/87e4ec8aa9fd49d9a108e5ad327526d5 2023-02-11 00:54:28,726 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,728 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-2a58391c4d584d1c8911b65de42bfe99, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-4eba672f17fe4f24bfb2e7a9cfc147fd, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-db4ba8661809480bb72daaa4d99ae627, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2364f56374aa4a7f8860d49550f4a66c, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14-bottom] to archive 2023-02-11 00:54:28,728 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:28,730 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,731 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,732 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,733 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-2a58391c4d584d1c8911b65de42bfe99 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-2a58391c4d584d1c8911b65de42bfe99 2023-02-11 00:54:28,735 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-4eba672f17fe4f24bfb2e7a9cfc147fd to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-4eba672f17fe4f24bfb2e7a9cfc147fd 2023-02-11 00:54:28,736 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-db4ba8661809480bb72daaa4d99ae627 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-db4ba8661809480bb72daaa4d99ae627 2023-02-11 00:54:28,737 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2364f56374aa4a7f8860d49550f4a66c to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/2364f56374aa4a7f8860d49550f4a66c 2023-02-11 00:54:28,738 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-11 00:54:28,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:28,742 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2. 2023-02-11 00:54:28,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3672dd666d16ab8bcdec33d37c2608f2: Waiting for close lock at 1676076868717Running coprocessor pre-close hooks at 1676076868717Disabling compacts and flushes for region at 1676076868717Disabling writes for close at 1676076868717Writing region close event to WAL at 1676076868739 (+22 ms)Running coprocessor post-close hooks at 1676076868742 (+3 ms)Closed at 1676076868742 2023-02-11 00:54:28,743 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:28,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6f03a7215e0b89c529706aaf04e65832, disabling compactions & flushes 2023-02-11 00:54:28,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. after waiting 0 ms 2023-02-11 00:54:28,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,744 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=3672dd666d16ab8bcdec33d37c2608f2, regionState=CLOSED 2023-02-11 00:54:28,744 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076868744"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076868744"}]},"ts":"1676076868744"} 2023-02-11 00:54:28,745 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-cae7d5d1979e44e8a4ffa24492c5599a, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-b17ed9f4b97c41bfb77130869482f7dd, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/17fb51d03fd746b79bf4f5f6bf303760, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-32ffc6b68838469e945f38e9619a0a25] to archive 2023-02-11 00:54:28,745 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:28,747 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/8b9b58ea7c2445d19bfe95ff62fa5911.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-11 00:54:28,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 3672dd666d16ab8bcdec33d37c2608f2, server=jenkins-hbase4.apache.org,33641,1676076840386 in 177 msec 2023-02-11 00:54:28,748 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/bec049088568420b8208c146c781ff57.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=3672dd666d16ab8bcdec33d37c2608f2, UNASSIGN in 184 msec 2023-02-11 00:54:28,749 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a9eef095d97c466a84cf5e45c21bf31d.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,750 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/a7e09760cade4a90997ca6ef1b1c6178.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,751 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-cae7d5d1979e44e8a4ffa24492c5599a to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:28,752 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-b17ed9f4b97c41bfb77130869482f7dd to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:28,753 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/17fb51d03fd746b79bf4f5f6bf303760 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/17fb51d03fd746b79bf4f5f6bf303760 2023-02-11 00:54:28,754 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-32ffc6b68838469e945f38e9619a0a25 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:28,755 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-9d0630de832e4c57aeac9c6a951a1af4, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-f06fe317287248ed98874d1821f6e188, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/b4c95f93e29944a5bc8405dcc5215b6b, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-c1e64bae4e3a4671bcdd87256b5a0e2e] to archive 2023-02-11 00:54:28,755 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:54:28,757 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3632fceb68f54d9a996bbc3d741fe185.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,757 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/3c8c85c3c14f4a61a43fb0303037c270.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,758 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/2841f32f231f438fb3463411b9e59543.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,759 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/a7eb3d0116754d24be2a9080e88bfd14.e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,760 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-9d0630de832e4c57aeac9c6a951a1af4 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:28,761 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-f06fe317287248ed98874d1821f6e188 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:28,761 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/b4c95f93e29944a5bc8405dcc5215b6b to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/b4c95f93e29944a5bc8405dcc5215b6b 2023-02-11 00:54:28,762 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-c1e64bae4e3a4671bcdd87256b5a0e2e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/testSplitTableRegionDeletedRowsDaughter=e3beb69538b330eb0b73628e03f86d35-c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:28,765 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-11 00:54:28,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:28,766 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832. 2023-02-11 00:54:28,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6f03a7215e0b89c529706aaf04e65832: Waiting for close lock at 1676076868744Running coprocessor pre-close hooks at 1676076868744Disabling compacts and flushes for region at 1676076868744Disabling writes for close at 1676076868744Writing region close event to WAL at 1676076868763 (+19 ms)Running coprocessor post-close hooks at 1676076868766 (+3 ms)Closed at 1676076868766 2023-02-11 00:54:28,767 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:28,767 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=6f03a7215e0b89c529706aaf04e65832, regionState=CLOSED 2023-02-11 00:54:28,767 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676076868767"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076868767"}]},"ts":"1676076868767"} 2023-02-11 00:54:28,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-11 00:54:28,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 6f03a7215e0b89c529706aaf04e65832, server=jenkins-hbase4.apache.org,33641,1676076840386 in 203 msec 2023-02-11 00:54:28,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-11 00:54:28,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=6f03a7215e0b89c529706aaf04e65832, UNASSIGN in 207 msec 2023-02-11 00:54:28,771 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076868771"}]},"ts":"1676076868771"} 2023-02-11 00:54:28,772 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-11 00:54:28,774 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-11 00:54:28,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 217 msec 2023-02-11 00:54:28,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 00:54:28,862 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-11 00:54:28,862 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,865 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,865 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-11 00:54:28,869 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,869 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,869 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:28,871 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/recovered.edits] 2023-02-11 00:54:28,871 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/recovered.edits] 2023-02-11 00:54:28,871 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/recovered.edits] 2023-02-11 00:54:28,874 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a7f6e0c983dd4ef199cabf0960ea3e7c to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf1/a7f6e0c983dd4ef199cabf0960ea3e7c 2023-02-11 00:54:28,874 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/6d7d2be88cbf45f3b9f765e55d2bc6a0 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf1/6d7d2be88cbf45f3b9f765e55d2bc6a0 2023-02-11 00:54:28,874 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/0dd43a93d5d04d2cb68aa4692d9cb92e 2023-02-11 00:54:28,876 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/32ffc6b68838469e945f38e9619a0a25 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/32ffc6b68838469e945f38e9619a0a25 2023-02-11 00:54:28,877 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/429b389ca60b428b92e13f7a0da5a38c to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/cf2/429b389ca60b428b92e13f7a0da5a38c 2023-02-11 00:54:28,877 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/0bb9f1c9b5b24fe8b4318a63491cb733 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/cf2/0bb9f1c9b5b24fe8b4318a63491cb733 2023-02-11 00:54:28,877 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b255d52cfcf4caf895d1e0d06cad976 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b255d52cfcf4caf895d1e0d06cad976 2023-02-11 00:54:28,878 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/8b9b58ea7c2445d19bfe95ff62fa5911 2023-02-11 00:54:28,879 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/recovered.edits/113.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2/recovered.edits/113.seqid 2023-02-11 00:54:28,879 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/recovered.edits/113.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832/recovered.edits/113.seqid 2023-02-11 00:54:28,880 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a7e09760cade4a90997ca6ef1b1c6178 2023-02-11 00:54:28,880 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:54:28,880 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/3672dd666d16ab8bcdec33d37c2608f2 2023-02-11 00:54:28,881 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/a9eef095d97c466a84cf5e45c21bf31d 2023-02-11 00:54:28,882 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/b17ed9f4b97c41bfb77130869482f7dd to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/b17ed9f4b97c41bfb77130869482f7dd 2023-02-11 00:54:28,883 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/bec049088568420b8208c146c781ff57 2023-02-11 00:54:28,884 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/cae7d5d1979e44e8a4ffa24492c5599a to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/cae7d5d1979e44e8a4ffa24492c5599a 2023-02-11 00:54:28,885 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/d72cfc4b8c1b402c933c598dd6fd8195 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf1/d72cfc4b8c1b402c933c598dd6fd8195 2023-02-11 00:54:28,887 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2841f32f231f438fb3463411b9e59543 2023-02-11 00:54:28,889 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2a58391c4d584d1c8911b65de42bfe99 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/2a58391c4d584d1c8911b65de42bfe99 2023-02-11 00:54:28,890 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3632fceb68f54d9a996bbc3d741fe185 2023-02-11 00:54:28,891 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/3c8c85c3c14f4a61a43fb0303037c270 2023-02-11 00:54:28,892 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/4eba672f17fe4f24bfb2e7a9cfc147fd 2023-02-11 00:54:28,893 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/9d0630de832e4c57aeac9c6a951a1af4 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/9d0630de832e4c57aeac9c6a951a1af4 2023-02-11 00:54:28,894 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/a7eb3d0116754d24be2a9080e88bfd14 2023-02-11 00:54:28,895 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/c1e64bae4e3a4671bcdd87256b5a0e2e 2023-02-11 00:54:28,896 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/db4ba8661809480bb72daaa4d99ae627 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/db4ba8661809480bb72daaa4d99ae627 2023-02-11 00:54:28,897 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/f06fe317287248ed98874d1821f6e188 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/cf2/f06fe317287248ed98874d1821f6e188 2023-02-11 00:54:28,899 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/recovered.edits/106.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35/recovered.edits/106.seqid 2023-02-11 00:54:28,899 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e3beb69538b330eb0b73628e03f86d35 2023-02-11 00:54:28,899 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-11 00:54:28,901 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,902 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-11 00:54:28,904 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-11 00:54:28,905 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,905 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-11 00:54:28,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076868905"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:28,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076868905"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:28,905 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076868905"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:28,907 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 00:54:28,907 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e3beb69538b330eb0b73628e03f86d35, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676076865052.e3beb69538b330eb0b73628e03f86d35.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 3672dd666d16ab8bcdec33d37c2608f2, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676076867153.3672dd666d16ab8bcdec33d37c2608f2.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 6f03a7215e0b89c529706aaf04e65832, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676076867153.6f03a7215e0b89c529706aaf04e65832.', STARTKEY => '60', ENDKEY => ''}] 2023-02-11 00:54:28,907 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-11 00:54:28,907 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076868907"}]},"ts":"9223372036854775807"} 2023-02-11 00:54:28,908 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-11 00:54:28,910 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:54:28,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 48 msec 2023-02-11 00:54:28,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-11 00:54:28,967 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-11 00:54:28,984 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=408 (was 404) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:38272 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:57268 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33904 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33890 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:38266 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:38298 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33934 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51890 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:35374 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:33916 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:38332 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:57274 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:35252 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d838b63-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:51906 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:57028 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1676793668_15 at /127.0.0.1:35422 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=871 (was 869) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=241 (was 193) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3185 (was 3257) 2023-02-11 00:54:28,999 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=408, OpenFileDescriptor=871, MaxFileDescriptor=60000, SystemLoadAverage=241, ProcessCount=168, AvailableMemoryMB=3184 2023-02-11 00:54:28,999 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:54:29,001 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:54:29,002 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:54:29,003 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:54:29,004 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:54:29,005 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:54:29,006 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,007 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 empty. 2023-02-11 00:54:29,007 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,007 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-11 00:54:29,021 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:29,022 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => f114625497f027c6d8b23a22cc933405, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:54:29,030 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:29,030 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing f114625497f027c6d8b23a22cc933405, disabling compactions & flushes 2023-02-11 00:54:29,030 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,030 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,030 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. after waiting 0 ms 2023-02-11 00:54:29,031 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,031 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,031 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for f114625497f027c6d8b23a22cc933405: Waiting for close lock at 1676076869030Disabling compacts and flushes for region at 1676076869030Disabling writes for close at 1676076869031 (+1 ms)Writing region close event to WAL at 1676076869031Closed at 1676076869031 2023-02-11 00:54:29,032 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:54:29,033 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076869033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076869033"}]},"ts":"1676076869033"} 2023-02-11 00:54:29,034 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:54:29,035 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:54:29,035 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076869035"}]},"ts":"1676076869035"} 2023-02-11 00:54:29,036 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-11 00:54:29,040 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:29,040 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:29,040 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:29,040 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:29,040 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:54:29,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, ASSIGN}] 2023-02-11 00:54:29,042 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, ASSIGN 2023-02-11 00:54:29,042 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:54:29,193 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:29,195 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=f114625497f027c6d8b23a22cc933405, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:29,195 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076869195"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076869195"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076869195"}]},"ts":"1676076869195"} 2023-02-11 00:54:29,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure f114625497f027c6d8b23a22cc933405, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:29,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f114625497f027c6d8b23a22cc933405, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:29,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:29,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:29,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,352 INFO [StoreOpener-f114625497f027c6d8b23a22cc933405-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,353 DEBUG [StoreOpener-f114625497f027c6d8b23a22cc933405-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/cf1 2023-02-11 00:54:29,353 DEBUG [StoreOpener-f114625497f027c6d8b23a22cc933405-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/cf1 2023-02-11 00:54:29,354 INFO [StoreOpener-f114625497f027c6d8b23a22cc933405-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f114625497f027c6d8b23a22cc933405 columnFamilyName cf1 2023-02-11 00:54:29,354 INFO [StoreOpener-f114625497f027c6d8b23a22cc933405-1] regionserver.HStore(310): Store=f114625497f027c6d8b23a22cc933405/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:29,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:54:29,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f114625497f027c6d8b23a22cc933405; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67880905, jitterRate=0.011504307389259338}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 00:54:29,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f114625497f027c6d8b23a22cc933405: Running coprocessor pre-open hook at 1676076869351Writing region info on filesystem at 1676076869351Initializing all the Stores at 1676076869352 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076869352Cleaning up temporary data from old regions at 1676076869355 (+3 ms)Cleaning up detritus from prior splits at 1676076869356 (+1 ms)Running coprocessor post-open hooks at 1676076869360 (+4 ms)Region opened successfully at 1676076869360 2023-02-11 00:54:29,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405., pid=87, masterSystemTime=1676076869348 2023-02-11 00:54:29,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,362 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=f114625497f027c6d8b23a22cc933405, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:29,362 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076869362"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076869362"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076869362"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076869362"}]},"ts":"1676076869362"} 2023-02-11 00:54:29,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-11 00:54:29,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure f114625497f027c6d8b23a22cc933405, server=jenkins-hbase4.apache.org,38317,1676076840434 in 167 msec 2023-02-11 00:54:29,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-11 00:54:29,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, ASSIGN in 325 msec 2023-02-11 00:54:29,367 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:54:29,367 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076869367"}]},"ts":"1676076869367"} 2023-02-11 00:54:29,368 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-11 00:54:29,372 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:54:29,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 370 msec 2023-02-11 00:54:29,513 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 00:54:29,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:54:29,533 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-11 00:54:29,534 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-11 00:54:29,534 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-11 00:54:29,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-11 00:54:29,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, REOPEN/MOVE}] 2023-02-11 00:54:29,550 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, REOPEN/MOVE 2023-02-11 00:54:29,551 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f114625497f027c6d8b23a22cc933405, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:29,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076869551"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076869551"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076869551"}]},"ts":"1676076869551"} 2023-02-11 00:54:29,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure f114625497f027c6d8b23a22cc933405, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:29,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f114625497f027c6d8b23a22cc933405, disabling compactions & flushes 2023-02-11 00:54:29,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. after waiting 0 ms 2023-02-11 00:54:29,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-11 00:54:29,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:54:29,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:29,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f114625497f027c6d8b23a22cc933405: Waiting for close lock at 1676076869704Running coprocessor pre-close hooks at 1676076869704Disabling compacts and flushes for region at 1676076869704Disabling writes for close at 1676076869705 (+1 ms)Writing region close event to WAL at 1676076869706 (+1 ms)Running coprocessor post-close hooks at 1676076869709 (+3 ms)Closed at 1676076869709 2023-02-11 00:54:29,709 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: f114625497f027c6d8b23a22cc933405 to self. 2023-02-11 00:54:29,710 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:29,711 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f114625497f027c6d8b23a22cc933405, regionState=CLOSED 2023-02-11 00:54:29,711 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076869711"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076869711"}]},"ts":"1676076869711"} 2023-02-11 00:54:29,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-11 00:54:29,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure f114625497f027c6d8b23a22cc933405, server=jenkins-hbase4.apache.org,38317,1676076840434 in 160 msec 2023-02-11 00:54:29,714 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=true 2023-02-11 00:54:29,864 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:54:29,865 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f114625497f027c6d8b23a22cc933405, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:29,865 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076869864"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076869864"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076869864"}]},"ts":"1676076869864"} 2023-02-11 00:54:29,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure f114625497f027c6d8b23a22cc933405, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:54:30,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:30,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f114625497f027c6d8b23a22cc933405, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:54:30,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:30,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:30,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:30,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:30,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:30,024 INFO [StoreOpener-f114625497f027c6d8b23a22cc933405-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:30,025 DEBUG [StoreOpener-f114625497f027c6d8b23a22cc933405-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/cf1 2023-02-11 00:54:30,025 DEBUG [StoreOpener-f114625497f027c6d8b23a22cc933405-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/cf1 2023-02-11 00:54:30,025 INFO [StoreOpener-f114625497f027c6d8b23a22cc933405-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f114625497f027c6d8b23a22cc933405 columnFamilyName cf1 2023-02-11 00:54:30,026 INFO [StoreOpener-f114625497f027c6d8b23a22cc933405-1] regionserver.HStore(310): Store=f114625497f027c6d8b23a22cc933405/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:54:30,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:30,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:30,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f114625497f027c6d8b23a22cc933405 2023-02-11 00:54:30,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f114625497f027c6d8b23a22cc933405; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62921748, jitterRate=-0.06239289045333862}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 00:54:30,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f114625497f027c6d8b23a22cc933405: Running coprocessor pre-open hook at 1676076870023Writing region info on filesystem at 1676076870023Initializing all the Stores at 1676076870023Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076870024 (+1 ms)Cleaning up temporary data from old regions at 1676076870027 (+3 ms)Cleaning up detritus from prior splits at 1676076870028 (+1 ms)Running coprocessor post-open hooks at 1676076870030 (+2 ms)Region opened successfully at 1676076870030 2023-02-11 00:54:30,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405., pid=92, masterSystemTime=1676076870019 2023-02-11 00:54:30,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:30,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:54:30,032 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f114625497f027c6d8b23a22cc933405, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:54:30,032 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076870032"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076870032"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076870032"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076870032"}]},"ts":"1676076870032"} 2023-02-11 00:54:30,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-11 00:54:30,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure f114625497f027c6d8b23a22cc933405, server=jenkins-hbase4.apache.org,38317,1676076840434 in 167 msec 2023-02-11 00:54:30,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-11 00:54:30,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, REOPEN/MOVE in 486 msec 2023-02-11 00:54:30,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-11 00:54:30,039 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 501 msec 2023-02-11 00:54:30,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6, ASSIGN}] 2023-02-11 00:54:30,042 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6, ASSIGN 2023-02-11 00:54:30,043 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 00:54:30,193 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:54:30,194 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:54:30,194 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:54:30,194 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:54:30,194 DEBUG [jenkins-hbase4:46819] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-11 00:54:30,195 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=6d7673c1d5364ef92bcb8dca986b97c6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:30,196 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076870195"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676076870195"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676076870195"}]},"ts":"1676076870195"} 2023-02-11 00:54:30,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 6d7673c1d5364ef92bcb8dca986b97c6, server=jenkins-hbase4.apache.org,41445,1676076840467}] 2023-02-11 00:54:30,352 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:54:30,352 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 00:54:30,353 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58430, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 00:54:30,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6. 2023-02-11 00:54:30,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6d7673c1d5364ef92bcb8dca986b97c6, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-11 00:54:30,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:54:30,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 6d7673c1d5364ef92bcb8dca986b97c6 2023-02-11 00:54:30,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:54:30,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6d7673c1d5364ef92bcb8dca986b97c6 2023-02-11 00:54:30,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6d7673c1d5364ef92bcb8dca986b97c6 2023-02-11 00:54:30,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:30,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:30,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:30,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:30,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:30,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:30,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:31,887 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-11 00:54:31,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,851 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 00:54:32,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:32,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:33,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:34,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:35,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:36,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:37,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:38,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:39,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:40,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:41,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:42,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:43,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:44,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:45,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:46,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:47,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:48,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:49,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:50,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:51,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:52,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:53,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:54,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:55,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:56,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:57,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:58,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:54:59,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:00,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:01,891 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 00:55:01,910 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 00:55:01,918 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 00:55:01,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:02,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:03,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:04,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:05,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:06,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:07,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:08,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:09,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:10,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:11,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:12,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:13,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:14,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:15,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:16,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:17,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:18,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:19,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:20,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:21,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:22,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:23,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:24,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:25,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:26,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:27,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:28,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 00:55:29,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 00:55:29,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:29,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 00:55:29,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 00:55:30,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 00:55:30,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:30,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,735 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:31,736 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:31,737 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:31,737 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:31,737 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:31,739 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting f114625497f027c6d8b23a22cc933405, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:31,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db exec-time=5 msec 2023-02-11 00:55:31,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:31,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 00:55:32,079 INFO [StoreOpener-6d7673c1d5364ef92bcb8dca986b97c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6d7673c1d5364ef92bcb8dca986b97c6 2023-02-11 00:55:32,083 DEBUG [StoreOpener-6d7673c1d5364ef92bcb8dca986b97c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/cf1 2023-02-11 00:55:32,083 DEBUG [StoreOpener-6d7673c1d5364ef92bcb8dca986b97c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/cf1 2023-02-11 00:55:32,084 INFO [StoreOpener-6d7673c1d5364ef92bcb8dca986b97c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6d7673c1d5364ef92bcb8dca986b97c6 columnFamilyName cf1 2023-02-11 00:55:32,085 INFO [StoreOpener-6d7673c1d5364ef92bcb8dca986b97c6-1] regionserver.HStore(310): Store=6d7673c1d5364ef92bcb8dca986b97c6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:32,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6d7673c1d5364ef92bcb8dca986b97c6; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67126564, jitterRate=2.637505531311035E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 00:55:32,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6d7673c1d5364ef92bcb8dca986b97c6: Running coprocessor pre-open hook at 1676076870359Initializing all the Stores at 1676076932078 (+61719 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076932078Running coprocessor post-open hooks at 1676076932086 (+8 ms)Region opened successfully at 1676076932086 2023-02-11 00:55:32,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6., pid=94, masterSystemTime=1676076870351 2023-02-11 00:55:32,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6. 2023-02-11 00:55:32,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6. 2023-02-11 00:55:32,089 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=6d7673c1d5364ef92bcb8dca986b97c6, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:32,090 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076932089"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676076932089"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676076932089"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676076932089"}]},"ts":"1676076932089"} 2023-02-11 00:55:32,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-11 00:55:32,093 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 6d7673c1d5364ef92bcb8dca986b97c6, server=jenkins-hbase4.apache.org,41445,1676076840467 in 1 mins, 1.894 sec 2023-02-11 00:55:32,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-11 00:55:32,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6, ASSIGN in 1 mins, 2.052 sec 2023-02-11 00:55:32,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.579 sec 2023-02-11 00:55:32,097 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676076869511","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62585,"client":"172.31.14.131:49384","queuetimems":0,"class":"HMaster"} 2023-02-11 00:55:32,097 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-11 00:55:33,987 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up 2023-02-11 00:55:33,988 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:33,990 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:33,990 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:33,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:33,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 00:55:33,994 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076933993"}]},"ts":"1676076933993"} 2023-02-11 00:55:33,995 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-11 00:55:33,996 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-11 00:55:33,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6, UNASSIGN}] 2023-02-11 00:55:33,997 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6, UNASSIGN 2023-02-11 00:55:33,998 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6d7673c1d5364ef92bcb8dca986b97c6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:33,998 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076933998"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676076933998"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676076933998"}]},"ts":"1676076933998"} 2023-02-11 00:55:33,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 6d7673c1d5364ef92bcb8dca986b97c6, server=jenkins-hbase4.apache.org,41445,1676076840467}] 2023-02-11 00:55:34,001 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, UNASSIGN 2023-02-11 00:55:34,001 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=f114625497f027c6d8b23a22cc933405, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,001 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076934001"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076934001"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076934001"}]},"ts":"1676076934001"} 2023-02-11 00:55:34,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure f114625497f027c6d8b23a22cc933405, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:34,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 00:55:34,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6d7673c1d5364ef92bcb8dca986b97c6 2023-02-11 00:55:34,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6d7673c1d5364ef92bcb8dca986b97c6, disabling compactions & flushes 2023-02-11 00:55:34,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6. 2023-02-11 00:55:34,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6. 2023-02-11 00:55:34,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6. after waiting 0 ms 2023-02-11 00:55:34,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6. 2023-02-11 00:55:34,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:34,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6. 2023-02-11 00:55:34,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6d7673c1d5364ef92bcb8dca986b97c6: Waiting for close lock at 1676076934151Running coprocessor pre-close hooks at 1676076934151Disabling compacts and flushes for region at 1676076934151Disabling writes for close at 1676076934152 (+1 ms)Writing region close event to WAL at 1676076934152Running coprocessor post-close hooks at 1676076934152Closed at 1676076934153 (+1 ms) 2023-02-11 00:55:34,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f114625497f027c6d8b23a22cc933405 2023-02-11 00:55:34,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f114625497f027c6d8b23a22cc933405, disabling compactions & flushes 2023-02-11 00:55:34,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:55:34,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:55:34,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. after waiting 0 ms 2023-02-11 00:55:34,155 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:55:34,155 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6d7673c1d5364ef92bcb8dca986b97c6 2023-02-11 00:55:34,155 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6d7673c1d5364ef92bcb8dca986b97c6, regionState=CLOSED 2023-02-11 00:55:34,155 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076934155"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676076934155"}]},"ts":"1676076934155"} 2023-02-11 00:55:34,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 00:55:34,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-11 00:55:34,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 6d7673c1d5364ef92bcb8dca986b97c6, server=jenkins-hbase4.apache.org,41445,1676076840467 in 159 msec 2023-02-11 00:55:34,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:34,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405. 2023-02-11 00:55:34,161 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f114625497f027c6d8b23a22cc933405: Waiting for close lock at 1676076934154Running coprocessor pre-close hooks at 1676076934154Disabling compacts and flushes for region at 1676076934154Disabling writes for close at 1676076934155 (+1 ms)Writing region close event to WAL at 1676076934155Running coprocessor post-close hooks at 1676076934161 (+6 ms)Closed at 1676076934161 2023-02-11 00:55:34,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6, UNASSIGN in 164 msec 2023-02-11 00:55:34,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f114625497f027c6d8b23a22cc933405 2023-02-11 00:55:34,163 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=f114625497f027c6d8b23a22cc933405, regionState=CLOSED 2023-02-11 00:55:34,163 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676076934163"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076934163"}]},"ts":"1676076934163"} 2023-02-11 00:55:34,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-11 00:55:34,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure f114625497f027c6d8b23a22cc933405, server=jenkins-hbase4.apache.org,38317,1676076840434 in 162 msec 2023-02-11 00:55:34,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-11 00:55:34,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f114625497f027c6d8b23a22cc933405, UNASSIGN in 169 msec 2023-02-11 00:55:34,168 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076934168"}]},"ts":"1676076934168"} 2023-02-11 00:55:34,169 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-11 00:55:34,171 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-11 00:55:34,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 181 msec 2023-02-11 00:55:34,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 00:55:34,295 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-11 00:55:34,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:34,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:34,298 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:34,298 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:34,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-11 00:55:34,303 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 2023-02-11 00:55:34,305 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/recovered.edits] 2023-02-11 00:55:34,310 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/recovered.edits/7.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405/recovered.edits/7.seqid 2023-02-11 00:55:34,311 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f114625497f027c6d8b23a22cc933405 2023-02-11 00:55:34,311 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-11 00:55:34,312 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:34,314 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-11 00:55:34,316 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-11 00:55:34,317 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:34,317 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-11 00:55:34,317 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076934317"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:34,317 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076934317"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:34,319 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-11 00:55:34,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f114625497f027c6d8b23a22cc933405, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676076869002.f114625497f027c6d8b23a22cc933405.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6d7673c1d5364ef92bcb8dca986b97c6, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676076869002_0001.6d7673c1d5364ef92bcb8dca986b97c6.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-11 00:55:34,319 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-11 00:55:34,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076934319"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:34,321 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-11 00:55:34,322 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:34,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 26 msec 2023-02-11 00:55:34,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-11 00:55:34,400 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-11 00:55:34,412 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=343 (was 408), OpenFileDescriptor=656 (was 871), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=113 (was 241), ProcessCount=168 (was 168), AvailableMemoryMB=2980 (was 3184) 2023-02-11 00:55:34,424 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=344, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=113, ProcessCount=168, AvailableMemoryMB=2979 2023-02-11 00:55:34,424 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:34,427 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:55:34,428 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:55:34,429 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:34,430 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:55:34,431 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:55:34,432 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,433 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365 empty. 2023-02-11 00:55:34,433 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,433 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-11 00:55:34,451 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-11 00:55:34,452 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6c5815c228f7755d66264b1163236365, NAME => 'testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:55:34,458 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:34,458 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 6c5815c228f7755d66264b1163236365, disabling compactions & flushes 2023-02-11 00:55:34,458 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,458 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,458 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. after waiting 0 ms 2023-02-11 00:55:34,458 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,459 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,459 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 6c5815c228f7755d66264b1163236365: Waiting for close lock at 1676076934458Disabling compacts and flushes for region at 1676076934458Disabling writes for close at 1676076934458Writing region close event to WAL at 1676076934458Closed at 1676076934459 (+1 ms) 2023-02-11 00:55:34,460 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:55:34,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676076934461"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076934461"}]},"ts":"1676076934461"} 2023-02-11 00:55:34,462 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:55:34,463 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:55:34,463 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076934463"}]},"ts":"1676076934463"} 2023-02-11 00:55:34,464 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-11 00:55:34,466 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:55:34,467 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:55:34,467 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:55:34,467 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:55:34,467 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:55:34,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=6c5815c228f7755d66264b1163236365, ASSIGN}] 2023-02-11 00:55:34,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=6c5815c228f7755d66264b1163236365, ASSIGN 2023-02-11 00:55:34,469 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=6c5815c228f7755d66264b1163236365, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:55:34,619 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:55:34,620 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=6c5815c228f7755d66264b1163236365, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,620 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676076934620"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076934620"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076934620"}]},"ts":"1676076934620"} 2023-02-11 00:55:34,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 6c5815c228f7755d66264b1163236365, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:34,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6c5815c228f7755d66264b1163236365, NAME => 'testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:55:34,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:34,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:34,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,778 INFO [StoreOpener-6c5815c228f7755d66264b1163236365-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,779 DEBUG [StoreOpener-6c5815c228f7755d66264b1163236365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1 2023-02-11 00:55:34,779 DEBUG [StoreOpener-6c5815c228f7755d66264b1163236365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1 2023-02-11 00:55:34,780 INFO [StoreOpener-6c5815c228f7755d66264b1163236365-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6c5815c228f7755d66264b1163236365 columnFamilyName cf1 2023-02-11 00:55:34,780 INFO [StoreOpener-6c5815c228f7755d66264b1163236365-1] regionserver.HStore(310): Store=6c5815c228f7755d66264b1163236365/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:34,780 INFO [StoreOpener-6c5815c228f7755d66264b1163236365-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,782 DEBUG [StoreOpener-6c5815c228f7755d66264b1163236365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2 2023-02-11 00:55:34,782 DEBUG [StoreOpener-6c5815c228f7755d66264b1163236365-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2 2023-02-11 00:55:34,782 INFO [StoreOpener-6c5815c228f7755d66264b1163236365-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6c5815c228f7755d66264b1163236365 columnFamilyName cf2 2023-02-11 00:55:34,783 INFO [StoreOpener-6c5815c228f7755d66264b1163236365-1] regionserver.HStore(310): Store=6c5815c228f7755d66264b1163236365/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:34,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:34,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6c5815c228f7755d66264b1163236365 2023-02-11 00:55:34,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:55:34,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6c5815c228f7755d66264b1163236365; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60794407, jitterRate=-0.09409274160861969}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:34,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6c5815c228f7755d66264b1163236365: Running coprocessor pre-open hook at 1676076934777Writing region info on filesystem at 1676076934777Initializing all the Stores at 1676076934778 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076934778Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076934778Cleaning up temporary data from old regions at 1676076934785 (+7 ms)Cleaning up detritus from prior splits at 1676076934785Running coprocessor post-open hooks at 1676076934789 (+4 ms)Region opened successfully at 1676076934789 2023-02-11 00:55:34,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365., pid=104, masterSystemTime=1676076934773 2023-02-11 00:55:34,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,792 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=6c5815c228f7755d66264b1163236365, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,792 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676076934792"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076934792"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076934792"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076934792"}]},"ts":"1676076934792"} 2023-02-11 00:55:34,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-11 00:55:34,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 6c5815c228f7755d66264b1163236365, server=jenkins-hbase4.apache.org,38317,1676076840434 in 172 msec 2023-02-11 00:55:34,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-11 00:55:34,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=6c5815c228f7755d66264b1163236365, ASSIGN in 328 msec 2023-02-11 00:55:34,797 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:55:34,797 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076934797"}]},"ts":"1676076934797"} 2023-02-11 00:55:34,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-11 00:55:34,801 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:55:34,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 373 msec 2023-02-11 00:55:34,937 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:34,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 00:55:34,938 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 00:55:34,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:34,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 00:55:34,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 00:55:34,939 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:34,939 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:34,945 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:34,945 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:34,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:34,945 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:34,945 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:34,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:34,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:34,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:34,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:34,945 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,945 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:34,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:34,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:34,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:34,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:34,946 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:34,947 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:34,947 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:34,947 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. started... 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:34,947 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:34,948 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6c5815c228f7755d66264b1163236365 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 00:55:34,949 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,950 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,950 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,950 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:34,950 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,950 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:34,962 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/35b55ec428e44f89adf1ebf555b1c341 2023-02-11 00:55:34,978 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/547598844d694dffb644259c0cf108f1 2023-02-11 00:55:34,982 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/35b55ec428e44f89adf1ebf555b1c341 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/35b55ec428e44f89adf1ebf555b1c341 2023-02-11 00:55:34,986 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/35b55ec428e44f89adf1ebf555b1c341, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:34,987 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/547598844d694dffb644259c0cf108f1 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/547598844d694dffb644259c0cf108f1 2023-02-11 00:55:34,990 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/547598844d694dffb644259c0cf108f1, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:34,991 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6c5815c228f7755d66264b1163236365 in 43ms, sequenceid=6, compaction requested=false 2023-02-11 00:55:34,991 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-11 00:55:34,991 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6c5815c228f7755d66264b1163236365: 2023-02-11 00:55:34,992 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:34,992 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:34,992 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:34,992 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,992 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:34,992 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:34,993 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,993 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:34,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:34,994 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,994 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:34,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:34,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:34,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:34,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:34,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:34,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:34,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 00:55:34,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76cd4d54[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:34,996 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 00:55:34,996 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:34,998 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:34,998 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:34,998 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,000 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:35,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 00:55:35,003 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 00:55:35,003 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:35,003 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:35,003 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 00:55:35,003 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:35,007 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,008 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,008 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,008 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,008 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,009 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,010 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,011 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,011 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,013 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,013 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,013 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,013 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,013 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 00:55:35,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,013 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:35,013 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,013 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,013 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,013 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,013 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:35,014 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:35,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,114 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:35,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:35,126 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:35,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:35,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 00:55:35,127 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 00:55:35,127 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,127 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 00:55:35,127 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 00:55:35,128 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,128 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,130 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,130 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,130 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,130 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,131 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,132 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,132 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,132 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,132 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,132 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,132 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,133 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:35,133 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:35,133 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,133 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. started... 2023-02-11 00:55:35,133 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6c5815c228f7755d66264b1163236365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:35,136 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,136 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,137 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,137 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,137 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,137 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,141 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/77910f318f474c33ad6847cd10a8c3d4 2023-02-11 00:55:35,157 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/6d2172a0c7654e4192fcfc2c46f9bda9 2023-02-11 00:55:35,161 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/77910f318f474c33ad6847cd10a8c3d4 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/77910f318f474c33ad6847cd10a8c3d4 2023-02-11 00:55:35,165 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/77910f318f474c33ad6847cd10a8c3d4, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:55:35,166 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/6d2172a0c7654e4192fcfc2c46f9bda9 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/6d2172a0c7654e4192fcfc2c46f9bda9 2023-02-11 00:55:35,171 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/6d2172a0c7654e4192fcfc2c46f9bda9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:55:35,172 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c5815c228f7755d66264b1163236365 in 39ms, sequenceid=19, compaction requested=false 2023-02-11 00:55:35,173 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6c5815c228f7755d66264b1163236365: 2023-02-11 00:55:35,173 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,173 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:35,173 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:35,173 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,173 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,173 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,174 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,174 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,175 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,175 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,177 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 00:55:35,177 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e9c4fee[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:35,177 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 00:55:35,177 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,178 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,178 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,178 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,179 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,179 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,179 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:35,180 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,180 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,180 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,180 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,180 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,180 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,182 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,182 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:35,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 00:55:35,186 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 00:55:35,186 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:35,186 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:35,186 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 00:55:35,186 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:35,188 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,188 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,188 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,188 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,189 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,189 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,189 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,191 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,191 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,192 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,194 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,194 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,194 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,195 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,195 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 00:55:35,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,195 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,195 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:35,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,196 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:35,196 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:35,296 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:35,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:35,308 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:35,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:35,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 00:55:35,310 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 00:55:35,310 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,310 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 00:55:35,310 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 00:55:35,310 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,310 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,311 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,311 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,311 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,311 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,311 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,312 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:35,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,313 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,313 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,314 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,314 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:35,314 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:35,314 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,314 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. started... 2023-02-11 00:55:35,314 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6c5815c228f7755d66264b1163236365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:35,316 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,316 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,316 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,316 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,317 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,317 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,324 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/4939020c6f3f4eaf82e6a40ed7d80d18 2023-02-11 00:55:35,335 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/81c80b063bd647d3881379d4bc9ab160 2023-02-11 00:55:35,339 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/4939020c6f3f4eaf82e6a40ed7d80d18 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/4939020c6f3f4eaf82e6a40ed7d80d18 2023-02-11 00:55:35,343 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/4939020c6f3f4eaf82e6a40ed7d80d18, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:55:35,344 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/81c80b063bd647d3881379d4bc9ab160 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/81c80b063bd647d3881379d4bc9ab160 2023-02-11 00:55:35,348 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/81c80b063bd647d3881379d4bc9ab160, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:55:35,349 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c5815c228f7755d66264b1163236365 in 34ms, sequenceid=32, compaction requested=true 2023-02-11 00:55:35,349 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6c5815c228f7755d66264b1163236365: 2023-02-11 00:55:35,349 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,349 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:35,349 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:35,349 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,349 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,349 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,350 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,350 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,351 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,351 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 00:55:35,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7871e3bb[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:35,353 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 00:55:35,353 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,354 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,354 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:35,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,354 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,354 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,354 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,354 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:35,354 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,354 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,355 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:35,355 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:35,357 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,357 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,357 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,358 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:35,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 00:55:35,362 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 00:55:35,362 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:35,362 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:35,362 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 00:55:35,362 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:35,363 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,363 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,363 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,364 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,364 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,365 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,365 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,366 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,368 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,368 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,368 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,374 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,374 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,374 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,374 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,375 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,374 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 00:55:35,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:35,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,374 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,375 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:35,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,375 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:35,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,375 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,376 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,475 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:35,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:35,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:35,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:35,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 00:55:35,489 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 00:55:35,489 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,489 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 00:55:35,489 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 00:55:35,489 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,489 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,491 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,491 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,491 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,491 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,491 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,491 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:35,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,491 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,492 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,492 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,492 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,492 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,492 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,492 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:35,493 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,493 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,493 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. started... 2023-02-11 00:55:35,493 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6c5815c228f7755d66264b1163236365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:35,494 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,494 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,494 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,496 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,496 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,496 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,501 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/c44155103ead4627b8a33d46f0b67d2a 2023-02-11 00:55:35,511 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/06d41a8bc381401a87f52a1333b72eb7 2023-02-11 00:55:35,514 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/c44155103ead4627b8a33d46f0b67d2a as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/c44155103ead4627b8a33d46f0b67d2a 2023-02-11 00:55:35,517 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/c44155103ead4627b8a33d46f0b67d2a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:55:35,518 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/06d41a8bc381401a87f52a1333b72eb7 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/06d41a8bc381401a87f52a1333b72eb7 2023-02-11 00:55:35,523 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/06d41a8bc381401a87f52a1333b72eb7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:55:35,524 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c5815c228f7755d66264b1163236365 in 31ms, sequenceid=45, compaction requested=true 2023-02-11 00:55:35,524 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6c5815c228f7755d66264b1163236365: 2023-02-11 00:55:35,524 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,524 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:35,524 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:35,524 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,524 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,524 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,526 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,526 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,526 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,526 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 00:55:35,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4140dbad[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:35,528 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 00:55:35,528 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,529 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,529 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,529 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,529 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,529 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,530 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,530 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:35,530 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,530 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:35,530 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,530 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,530 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:35,530 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,530 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,532 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,532 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,532 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,532 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:35,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 00:55:35,536 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 00:55:35,536 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:35,536 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:35,536 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 00:55:35,536 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:35,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,538 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,538 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,538 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,538 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,540 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,540 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,541 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,543 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 00:55:35,543 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,543 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:35,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,543 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,544 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,644 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:35,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:35,655 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:35,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:35,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 00:55:35,657 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 00:55:35,657 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,657 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 00:55:35,657 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 00:55:35,657 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,657 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,658 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,658 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,658 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,658 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,659 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,659 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,659 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:35,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,659 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,660 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,660 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,660 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,660 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,660 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,661 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,661 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,661 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,661 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,661 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:35,661 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:35,661 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,661 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. started... 2023-02-11 00:55:35,661 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6c5815c228f7755d66264b1163236365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,663 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,666 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,666 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,669 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/66fb6cc4354c43278dc4384b175cb2f8 2023-02-11 00:55:35,682 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/e98c05dcaefd4271b52baa9e96a55e9d 2023-02-11 00:55:35,686 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/66fb6cc4354c43278dc4384b175cb2f8 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/66fb6cc4354c43278dc4384b175cb2f8 2023-02-11 00:55:35,690 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/66fb6cc4354c43278dc4384b175cb2f8, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:55:35,690 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/e98c05dcaefd4271b52baa9e96a55e9d as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/e98c05dcaefd4271b52baa9e96a55e9d 2023-02-11 00:55:35,695 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/e98c05dcaefd4271b52baa9e96a55e9d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:55:35,696 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c5815c228f7755d66264b1163236365 in 35ms, sequenceid=58, compaction requested=true 2023-02-11 00:55:35,696 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6c5815c228f7755d66264b1163236365: 2023-02-11 00:55:35,696 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,696 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:35,696 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:35,696 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,696 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,696 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,698 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 00:55:35,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5dfb893c[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:35,700 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 00:55:35,700 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,701 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,701 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,701 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,702 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,702 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,702 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:35,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,707 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,710 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,710 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,710 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,710 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,710 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,710 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:35,714 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 00:55:35,714 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 00:55:35,714 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:35,714 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:35,714 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 00:55:35,714 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:35,715 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,715 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,717 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,717 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,718 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,719 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,720 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,720 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,725 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,725 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,724 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,724 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,724 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 00:55:35,725 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,725 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:35,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,725 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:35,826 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:35,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:35,841 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:35,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:35,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 00:55:35,842 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 00:55:35,842 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,842 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 00:55:35,842 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 00:55:35,843 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,843 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,845 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,845 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,845 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,846 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,847 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:35,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,847 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 00:55:35,847 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,847 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,847 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,847 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:35,847 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,848 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. started... 2023-02-11 00:55:35,847 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,848 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6c5815c228f7755d66264b1163236365 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:35,849 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,849 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,849 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,849 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,849 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,850 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,861 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/d2743e1e43e449cb802b1bd4e74366a9 2023-02-11 00:55:35,875 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/fe339cdc6827446ab3e4e10c8770d906 2023-02-11 00:55:35,879 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/d2743e1e43e449cb802b1bd4e74366a9 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/d2743e1e43e449cb802b1bd4e74366a9 2023-02-11 00:55:35,883 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/d2743e1e43e449cb802b1bd4e74366a9, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:55:35,884 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/fe339cdc6827446ab3e4e10c8770d906 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/fe339cdc6827446ab3e4e10c8770d906 2023-02-11 00:55:35,889 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/fe339cdc6827446ab3e4e10c8770d906, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:55:35,890 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c5815c228f7755d66264b1163236365 in 42ms, sequenceid=71, compaction requested=true 2023-02-11 00:55:35,890 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6c5815c228f7755d66264b1163236365: 2023-02-11 00:55:35,890 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:35,890 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:35,890 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:35,890 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,890 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 00:55:35,890 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 00:55:35,892 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,892 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,892 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,892 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:35,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 00:55:35,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71b44e52[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:35,898 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 00:55:35,898 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,899 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:35,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,899 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,899 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,899 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,899 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:35,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,900 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 00:55:35,900 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 00:55:35,900 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:35,899 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:35,902 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,902 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,902 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,902 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,902 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,902 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,902 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,903 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 00:55:35,903 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,903 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 00:55:35,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:35,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 00:55:35,906 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 00:55:35,906 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:35,906 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:35,906 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 00:55:35,906 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:35,908 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,908 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:35,908 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,908 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,908 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:35,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,910 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:35,911 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,911 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,912 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,912 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:35,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 00:55:35,913 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,913 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,914 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,917 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,917 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,918 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:35,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,918 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 00:55:35,918 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 00:55:35,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 00:55:35,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:35,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:35,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:35,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:35,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:36,019 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:36,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:36,030 INFO [Listener at localhost/35701] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:36,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.StoreUtils(129): cannot split hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/77910f318f474c33ad6847cd10a8c3d4 because midkey is the same as first or last row 2023-02-11 00:55:36,031 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.StoreUtils(129): cannot split hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/6d2172a0c7654e4192fcfc2c46f9bda9 because midkey is the same as first or last row 2023-02-11 00:55:36,031 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.HRegion(2744): Flushing 6c5815c228f7755d66264b1163236365 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 00:55:36,043 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/55f60e677d0545779c9f547d5f499e93 2023-02-11 00:55:36,055 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/442c0118e40846fc8fe78b7e6fe886aa 2023-02-11 00:55:36,059 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf1/55f60e677d0545779c9f547d5f499e93 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/55f60e677d0545779c9f547d5f499e93 2023-02-11 00:55:36,064 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/55f60e677d0545779c9f547d5f499e93, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:55:36,064 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/.tmp/cf2/442c0118e40846fc8fe78b7e6fe886aa as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/442c0118e40846fc8fe78b7e6fe886aa 2023-02-11 00:55:36,069 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/442c0118e40846fc8fe78b7e6fe886aa, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:55:36,070 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6c5815c228f7755d66264b1163236365 in 38ms, sequenceid=82, compaction requested=true 2023-02-11 00:55:36,070 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.HRegion(2445): Flush status journal for 6c5815c228f7755d66264b1163236365: 2023-02-11 00:55:36,071 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.StoreUtils(129): cannot split hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/77910f318f474c33ad6847cd10a8c3d4 because midkey is the same as first or last row 2023-02-11 00:55:36,071 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38317] regionserver.StoreUtils(129): cannot split hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/6d2172a0c7654e4192fcfc2c46f9bda9 because midkey is the same as first or last row 2023-02-11 00:55:36,071 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:36,071 DEBUG [Listener at localhost/35701] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-11 00:55:36,071 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:36,073 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-11 00:55:36,073 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-11 00:55:36,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:36,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 00:55:36,078 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076936077"}]},"ts":"1676076936077"} 2023-02-11 00:55:36,079 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-11 00:55:36,080 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-11 00:55:36,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=6c5815c228f7755d66264b1163236365, UNASSIGN}] 2023-02-11 00:55:36,081 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=6c5815c228f7755d66264b1163236365, UNASSIGN 2023-02-11 00:55:36,082 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6c5815c228f7755d66264b1163236365, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:36,082 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676076936082"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076936082"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076936082"}]},"ts":"1676076936082"} 2023-02-11 00:55:36,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 6c5815c228f7755d66264b1163236365, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:36,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 00:55:36,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6c5815c228f7755d66264b1163236365 2023-02-11 00:55:36,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6c5815c228f7755d66264b1163236365, disabling compactions & flushes 2023-02-11 00:55:36,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:36,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:36,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. after waiting 0 ms 2023-02-11 00:55:36,235 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:36,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 00:55:36,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:36,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365. 2023-02-11 00:55:36,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6c5815c228f7755d66264b1163236365: Waiting for close lock at 1676076936235Running coprocessor pre-close hooks at 1676076936235Disabling compacts and flushes for region at 1676076936235Disabling writes for close at 1676076936235Writing region close event to WAL at 1676076936236 (+1 ms)Running coprocessor post-close hooks at 1676076936239 (+3 ms)Closed at 1676076936239 2023-02-11 00:55:36,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6c5815c228f7755d66264b1163236365 2023-02-11 00:55:36,241 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6c5815c228f7755d66264b1163236365, regionState=CLOSED 2023-02-11 00:55:36,241 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676076936241"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076936241"}]},"ts":"1676076936241"} 2023-02-11 00:55:36,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-11 00:55:36,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 6c5815c228f7755d66264b1163236365, server=jenkins-hbase4.apache.org,38317,1676076840434 in 159 msec 2023-02-11 00:55:36,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-11 00:55:36,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=6c5815c228f7755d66264b1163236365, UNASSIGN in 162 msec 2023-02-11 00:55:36,245 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076936245"}]},"ts":"1676076936245"} 2023-02-11 00:55:36,246 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-11 00:55:36,247 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-11 00:55:36,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 174 msec 2023-02-11 00:55:36,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 00:55:36,379 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-11 00:55:36,380 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-11 00:55:36,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:36,383 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:36,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-11 00:55:36,383 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:36,386 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365 2023-02-11 00:55:36,388 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/recovered.edits] 2023-02-11 00:55:36,391 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/35b55ec428e44f89adf1ebf555b1c341 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/35b55ec428e44f89adf1ebf555b1c341 2023-02-11 00:55:36,392 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/4939020c6f3f4eaf82e6a40ed7d80d18 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/4939020c6f3f4eaf82e6a40ed7d80d18 2023-02-11 00:55:36,393 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/55f60e677d0545779c9f547d5f499e93 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/55f60e677d0545779c9f547d5f499e93 2023-02-11 00:55:36,394 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/66fb6cc4354c43278dc4384b175cb2f8 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/66fb6cc4354c43278dc4384b175cb2f8 2023-02-11 00:55:36,395 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/77910f318f474c33ad6847cd10a8c3d4 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/77910f318f474c33ad6847cd10a8c3d4 2023-02-11 00:55:36,396 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/c44155103ead4627b8a33d46f0b67d2a to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/c44155103ead4627b8a33d46f0b67d2a 2023-02-11 00:55:36,397 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/d2743e1e43e449cb802b1bd4e74366a9 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf1/d2743e1e43e449cb802b1bd4e74366a9 2023-02-11 00:55:36,399 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/06d41a8bc381401a87f52a1333b72eb7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/06d41a8bc381401a87f52a1333b72eb7 2023-02-11 00:55:36,400 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/442c0118e40846fc8fe78b7e6fe886aa to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/442c0118e40846fc8fe78b7e6fe886aa 2023-02-11 00:55:36,401 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/547598844d694dffb644259c0cf108f1 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/547598844d694dffb644259c0cf108f1 2023-02-11 00:55:36,402 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/6d2172a0c7654e4192fcfc2c46f9bda9 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/6d2172a0c7654e4192fcfc2c46f9bda9 2023-02-11 00:55:36,403 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/81c80b063bd647d3881379d4bc9ab160 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/81c80b063bd647d3881379d4bc9ab160 2023-02-11 00:55:36,404 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/e98c05dcaefd4271b52baa9e96a55e9d to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/e98c05dcaefd4271b52baa9e96a55e9d 2023-02-11 00:55:36,405 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/fe339cdc6827446ab3e4e10c8770d906 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/cf2/fe339cdc6827446ab3e4e10c8770d906 2023-02-11 00:55:36,407 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/recovered.edits/85.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365/recovered.edits/85.seqid 2023-02-11 00:55:36,408 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testInvalidSplitKey/6c5815c228f7755d66264b1163236365 2023-02-11 00:55:36,408 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-11 00:55:36,410 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:36,411 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-11 00:55:36,414 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-11 00:55:36,415 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:36,415 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-11 00:55:36,415 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076936415"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:36,417 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 00:55:36,417 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6c5815c228f7755d66264b1163236365, NAME => 'testInvalidSplitKey,,1676076934428.6c5815c228f7755d66264b1163236365.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 00:55:36,417 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-11 00:55:36,417 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076936417"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:36,418 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-11 00:55:36,420 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:36,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 40 msec 2023-02-11 00:55:36,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-11 00:55:36,485 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-11 00:55:36,498 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=366 (was 344) - Thread LEAK? -, OpenFileDescriptor=745 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=113 (was 113), ProcessCount=168 (was 168), AvailableMemoryMB=2969 (was 2979) 2023-02-11 00:55:36,514 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=366, OpenFileDescriptor=745, MaxFileDescriptor=60000, SystemLoadAverage=113, ProcessCount=168, AvailableMemoryMB=2969 2023-02-11 00:55:36,514 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:36,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:55:36,517 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:55:36,518 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:36,520 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:55:36,521 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:55:36,522 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,523 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931 empty. 2023-02-11 00:55:36,523 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,523 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-11 00:55:36,540 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-11 00:55:36,541 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3e2f92c6000f19c73dc861b8693fb931, NAME => 'testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:55:36,558 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:36,558 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 3e2f92c6000f19c73dc861b8693fb931, disabling compactions & flushes 2023-02-11 00:55:36,558 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:36,558 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:36,558 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. after waiting 0 ms 2023-02-11 00:55:36,558 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:36,558 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:36,558 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 3e2f92c6000f19c73dc861b8693fb931: Waiting for close lock at 1676076936558Disabling compacts and flushes for region at 1676076936558Disabling writes for close at 1676076936558Writing region close event to WAL at 1676076936558Closed at 1676076936558 2023-02-11 00:55:36,560 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:55:36,561 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076936561"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076936561"}]},"ts":"1676076936561"} 2023-02-11 00:55:36,562 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:55:36,563 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:55:36,563 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076936563"}]},"ts":"1676076936563"} 2023-02-11 00:55:36,564 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-11 00:55:36,566 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:55:36,566 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:55:36,567 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:55:36,567 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:55:36,567 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:55:36,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, ASSIGN}] 2023-02-11 00:55:36,568 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, ASSIGN 2023-02-11 00:55:36,568 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:55:36,719 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:55:36,720 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=3e2f92c6000f19c73dc861b8693fb931, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:36,720 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076936720"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076936720"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076936720"}]},"ts":"1676076936720"} 2023-02-11 00:55:36,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:36,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:36,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3e2f92c6000f19c73dc861b8693fb931, NAME => 'testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:55:36,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:36,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:36,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,877 INFO [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,878 DEBUG [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1 2023-02-11 00:55:36,878 DEBUG [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1 2023-02-11 00:55:36,878 INFO [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3e2f92c6000f19c73dc861b8693fb931 columnFamilyName cf1 2023-02-11 00:55:36,879 INFO [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] regionserver.HStore(310): Store=3e2f92c6000f19c73dc861b8693fb931/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:36,879 INFO [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,880 DEBUG [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2 2023-02-11 00:55:36,880 DEBUG [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2 2023-02-11 00:55:36,880 INFO [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3e2f92c6000f19c73dc861b8693fb931 columnFamilyName cf2 2023-02-11 00:55:36,880 INFO [StoreOpener-3e2f92c6000f19c73dc861b8693fb931-1] regionserver.HStore(310): Store=3e2f92c6000f19c73dc861b8693fb931/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:36,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:36,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:36,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:55:36,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3e2f92c6000f19c73dc861b8693fb931; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62392167, jitterRate=-0.07028426229953766}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:36,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3e2f92c6000f19c73dc861b8693fb931: Running coprocessor pre-open hook at 1676076936876Writing region info on filesystem at 1676076936876Initializing all the Stores at 1676076936876Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076936876Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076936876Cleaning up temporary data from old regions at 1676076936882 (+6 ms)Cleaning up detritus from prior splits at 1676076936882Running coprocessor post-open hooks at 1676076936885 (+3 ms)Region opened successfully at 1676076936885 2023-02-11 00:55:36,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931., pid=111, masterSystemTime=1676076936872 2023-02-11 00:55:36,888 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:36,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:36,888 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=3e2f92c6000f19c73dc861b8693fb931, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:36,889 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076936888"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076936888"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076936888"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076936888"}]},"ts":"1676076936888"} 2023-02-11 00:55:36,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-11 00:55:36,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 in 169 msec 2023-02-11 00:55:36,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-11 00:55:36,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, ASSIGN in 325 msec 2023-02-11 00:55:36,894 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:55:36,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076936894"}]},"ts":"1676076936894"} 2023-02-11 00:55:36,895 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-11 00:55:36,898 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:55:36,899 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 380 msec 2023-02-11 00:55:37,025 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:37,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 00:55:37,027 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,027 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,027 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 00:55:37,027 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:55:37,027 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,028 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,029 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,029 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,029 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,029 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,029 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,029 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:37,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,030 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,031 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,031 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,031 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,031 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:37,031 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:37,031 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,031 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. started... 2023-02-11 00:55:37,032 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3e2f92c6000f19c73dc861b8693fb931 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 00:55:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,033 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,042 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/1576d7857a5a4b60b2ec493cee34b75d 2023-02-11 00:55:37,056 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/8d779e79780842cca528df4a4fbbe4ed 2023-02-11 00:55:37,061 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/1576d7857a5a4b60b2ec493cee34b75d as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d 2023-02-11 00:55:37,065 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:37,066 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/8d779e79780842cca528df4a4fbbe4ed as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed 2023-02-11 00:55:37,069 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:37,070 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3e2f92c6000f19c73dc861b8693fb931 in 38ms, sequenceid=6, compaction requested=false 2023-02-11 00:55:37,070 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,071 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3e2f92c6000f19c73dc861b8693fb931: 2023-02-11 00:55:37,071 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,071 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:37,071 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:37,071 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,071 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,071 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,072 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,072 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,072 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,072 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 00:55:37,074 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6af17e4[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:37,074 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:55:37,074 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,076 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:37,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:37,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,076 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:37,077 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,077 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,077 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,078 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,078 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,078 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,077 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,078 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,078 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,078 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:37,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:55:37,081 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 00:55:37,081 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:37,081 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:37,081 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,081 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:37,082 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,082 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,082 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,082 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,082 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,083 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,083 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,083 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,083 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,084 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,085 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,085 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,086 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,088 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,088 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,088 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:37,088 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,089 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:37,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,089 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,089 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,089 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,089 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,089 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,089 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,089 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,186 ERROR [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-74375045-172.31.14.131-1676076836114 (Datanode Uuid e08011bb-ec6b-4682-9527-eafae3012242) service to localhost/127.0.0.1:46775 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 00:55:37,189 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:37,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:37,200 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:37,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:37,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 00:55:37,201 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,201 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,201 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 00:55:37,201 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:55:37,202 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,202 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,204 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,204 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,204 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,204 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,204 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,204 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,205 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,205 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:37,206 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,206 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,206 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. started... 2023-02-11 00:55:37,206 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3e2f92c6000f19c73dc861b8693fb931 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:37,207 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,208 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,208 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,208 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,208 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,208 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,215 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/5dded02a93114e76b118bdd920b6ce55 2023-02-11 00:55:37,225 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/bc3d93b722b241959473c39baf754e94 2023-02-11 00:55:37,228 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/5dded02a93114e76b118bdd920b6ce55 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55 2023-02-11 00:55:37,231 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:55:37,232 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/bc3d93b722b241959473c39baf754e94 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94 2023-02-11 00:55:37,235 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:55:37,236 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e2f92c6000f19c73dc861b8693fb931 in 30ms, sequenceid=19, compaction requested=false 2023-02-11 00:55:37,236 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3e2f92c6000f19c73dc861b8693fb931: 2023-02-11 00:55:37,236 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,236 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:37,236 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:37,236 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,236 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,236 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,241 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,241 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,242 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,242 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 00:55:37,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41f5392f[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:37,243 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:55:37,244 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,244 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,244 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:37,244 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,245 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,245 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,245 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:37,246 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,246 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,246 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,248 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,248 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,248 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,248 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,248 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,248 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,248 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:37,251 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:55:37,251 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 00:55:37,251 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:37,251 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:37,251 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,251 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:37,253 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,253 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,253 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,253 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,254 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,254 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,256 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,256 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,256 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,258 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,258 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,259 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:37,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,258 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,259 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:37,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,259 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,259 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,259 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,259 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,259 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,259 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,259 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,260 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,260 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,359 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:37,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:37,370 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:37,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:37,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 00:55:37,371 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,371 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,371 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 00:55:37,371 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:55:37,372 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,372 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,373 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,373 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,373 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,373 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,374 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,375 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:37,375 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:37,375 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,375 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. started... 2023-02-11 00:55:37,376 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3e2f92c6000f19c73dc861b8693fb931 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:37,377 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,378 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,378 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,378 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,378 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,378 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,383 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/55f52ffbe55444279268ca8905cbf75c 2023-02-11 00:55:37,393 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/c37556de43e7405e923aa0a0b43c569b 2023-02-11 00:55:37,396 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/55f52ffbe55444279268ca8905cbf75c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c 2023-02-11 00:55:37,399 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:55:37,400 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/c37556de43e7405e923aa0a0b43c569b as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b 2023-02-11 00:55:37,403 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:55:37,403 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e2f92c6000f19c73dc861b8693fb931 in 27ms, sequenceid=32, compaction requested=true 2023-02-11 00:55:37,404 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3e2f92c6000f19c73dc861b8693fb931: 2023-02-11 00:55:37,404 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,404 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:37,404 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:37,404 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,404 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,404 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,405 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,405 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 00:55:37,407 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4046fa44[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:37,407 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:55:37,407 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,409 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,409 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,409 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,409 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:37,409 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,409 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,409 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,409 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:37,409 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:37,409 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:37,411 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,411 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,411 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,411 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,412 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,412 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,412 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,412 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,412 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,412 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:37,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:55:37,414 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 00:55:37,414 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:37,414 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:37,415 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,415 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:37,416 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,416 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,416 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,416 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,417 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,417 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,418 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,419 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,419 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,425 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,426 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:37,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,426 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:37,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,426 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,427 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,427 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,427 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,427 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,427 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,427 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,526 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:37,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:37,537 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:37,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:37,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 00:55:37,539 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,539 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,539 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 00:55:37,539 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:55:37,539 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,539 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,540 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,540 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,540 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,540 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,541 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,541 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,542 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,542 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,542 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,542 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,542 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,542 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,543 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:37,543 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:37,543 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,543 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. started... 2023-02-11 00:55:37,543 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3e2f92c6000f19c73dc861b8693fb931 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:37,543 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,544 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,544 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,544 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,544 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,544 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,551 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/56839b6a9ad2464380c7ffc75ec8833a 2023-02-11 00:55:37,560 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/93485d29aa1449349e16411ec9417e96 2023-02-11 00:55:37,563 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/56839b6a9ad2464380c7ffc75ec8833a as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a 2023-02-11 00:55:37,566 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:55:37,567 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/93485d29aa1449349e16411ec9417e96 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96 2023-02-11 00:55:37,570 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:55:37,570 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e2f92c6000f19c73dc861b8693fb931 in 27ms, sequenceid=45, compaction requested=true 2023-02-11 00:55:37,570 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3e2f92c6000f19c73dc861b8693fb931: 2023-02-11 00:55:37,570 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,570 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:37,570 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:37,570 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,571 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,571 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,573 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,573 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,573 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,573 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 00:55:37,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1353fac0[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:37,575 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:55:37,575 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:37,576 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,576 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,576 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:37,576 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,577 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,577 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:37,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,577 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,577 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,577 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:37,578 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,578 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,578 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,578 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,578 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,578 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,578 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,580 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,580 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,580 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:37,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:55:37,583 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 00:55:37,583 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:37,583 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:37,583 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,583 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:37,584 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,584 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,584 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,584 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,585 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,585 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,586 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,587 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,587 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,589 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,590 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:37,589 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,589 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,590 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,590 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,590 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,590 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:37,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,590 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,590 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:37,590 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,591 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,591 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,691 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:37,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:37,702 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:37,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:37,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 00:55:37,703 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,703 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,703 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 00:55:37,703 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:55:37,704 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,704 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,705 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,705 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,705 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,705 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,705 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,705 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:37,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,707 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,707 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,707 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,707 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,707 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,707 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,707 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,707 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:37,707 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,707 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. started... 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,707 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,707 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3e2f92c6000f19c73dc861b8693fb931 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:37,710 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,711 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,711 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,711 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,711 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,711 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,716 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/22d1f5e7d3864399a120dfc24ba538f2 2023-02-11 00:55:37,726 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/55faf55c0afd44c1afa42756d93ec3e6 2023-02-11 00:55:37,729 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/22d1f5e7d3864399a120dfc24ba538f2 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2 2023-02-11 00:55:37,732 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:55:37,733 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/55faf55c0afd44c1afa42756d93ec3e6 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6 2023-02-11 00:55:37,736 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:55:37,736 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e2f92c6000f19c73dc861b8693fb931 in 29ms, sequenceid=58, compaction requested=true 2023-02-11 00:55:37,736 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3e2f92c6000f19c73dc861b8693fb931: 2023-02-11 00:55:37,736 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,737 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:37,737 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:37,737 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,737 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,737 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,740 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,740 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,741 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,741 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 00:55:37,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d3ddf79[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:37,742 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:55:37,742 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,744 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,744 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,744 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,744 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,744 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,744 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:37,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:37,746 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,746 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,746 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,746 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,746 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,746 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,746 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,746 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,746 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,747 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:37,749 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:55:37,749 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 00:55:37,749 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:37,749 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:37,749 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,749 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:37,751 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,751 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,751 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,751 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,752 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,752 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,754 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,754 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,754 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,756 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,756 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,756 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,756 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,757 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,756 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:37,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,757 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,758 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,758 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,758 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,858 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:37,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:37,869 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:37,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:37,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 00:55:37,871 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,871 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,871 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 00:55:37,871 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 00:55:37,873 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,873 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,875 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,875 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,875 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,875 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,876 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,876 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,877 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,877 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:37,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,878 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,878 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 00:55:37,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,879 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,879 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. started... 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,879 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3e2f92c6000f19c73dc861b8693fb931 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,879 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,881 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,888 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 2023-02-11 00:55:37,900 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/b2860b675b374dffb24da0aae58b8a0f 2023-02-11 00:55:37,904 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 2023-02-11 00:55:37,907 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:55:37,908 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/b2860b675b374dffb24da0aae58b8a0f as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f 2023-02-11 00:55:37,911 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:55:37,912 INFO [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3e2f92c6000f19c73dc861b8693fb931 in 33ms, sequenceid=71, compaction requested=true 2023-02-11 00:55:37,912 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3e2f92c6000f19c73dc861b8693fb931: 2023-02-11 00:55:37,912 DEBUG [rs(jenkins-hbase4.apache.org,38317,1676076840434)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:37,913 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:37,913 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:37,913 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,913 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 00:55:37,913 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 00:55:37,914 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,914 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,915 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,915 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:37,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 00:55:37,917 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@569afb0[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:37,917 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 00:55:37,917 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:37,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,918 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 00:55:37,918 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:37,921 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,921 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,921 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,921 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,922 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,922 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,922 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 00:55:37,922 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,922 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,922 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 00:55:37,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,38317,1676076840434': 2023-02-11 00:55:37,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 00:55:37,925 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 00:55:37,925 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:37,925 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:37,925 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 00:55:37,925 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:37,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:37,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,927 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:37,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,928 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,928 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:37,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,930 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:37,930 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 00:55:37,930 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:37,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:37,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:37,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:37,937 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:37,937 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 00:55:37,937 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:37,937 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:38,038 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:38,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:38,296 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 00:55:38,296 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:38,297 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,298 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,299 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,299 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,299 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:38,299 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:38,299 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:38,548 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,548 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:38,548 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:38,548 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:38,548 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-11 00:55:38,548 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:38,548 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:38,548 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:38,548 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:38,549 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:38,550 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 00:55:38,550 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:38,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:38,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:38,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:38,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:38,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:38,551 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:38,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:38,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:38,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:38,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:38,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:38,552 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:38,553 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:38,554 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:38,554 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:38,554 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:38,554 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:38,554 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:38,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:38,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:38,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:38,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:38,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:38,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:38,555 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:38,556 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,556 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:38,556 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,556 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,556 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,556 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:55:38,556 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:38,557 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:38,557 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076968557 2023-02-11 00:55:38,557 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:38,559 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,561 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:38,561 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:38,561 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:38,561 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076998561 2023-02-11 00:55:38,561 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077058561 2023-02-11 00:55:38,561 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:55:38,561 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:38,563 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:38,563 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:38,563 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:38,811 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,812 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:38,812 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:38,812 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:38,812 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:38,812 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:38,812 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:38,812 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:38,812 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:38,813 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:38,814 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 00:55:38,814 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:38,815 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:38,815 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:38,815 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:38,815 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:38,815 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:38,815 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:38,815 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:38,816 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:38,816 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:38,816 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:38,816 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:38,816 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:38,816 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:38,816 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:38,817 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:38,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:38,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:38,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:38,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:38,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:38,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:38,818 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:38,819 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:38,819 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:38,819 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:38,819 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,819 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:38,819 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,819 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,819 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,820 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:55:38,820 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:38,821 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:38,821 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076968821 2023-02-11 00:55:38,821 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:38,822 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,825 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:38,825 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:38,825 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:38,825 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076998825 2023-02-11 00:55:38,825 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077058825 2023-02-11 00:55:38,825 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 00:55:38,825 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:38,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN}] 2023-02-11 00:55:38,827 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:38,827 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:38,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:38,827 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:39,075 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,075 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:39,075 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:39,075 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:39,076 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:39,076 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:39,076 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:39,076 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:39,076 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:39,076 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:39,078 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 00:55:39,078 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:39,078 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:39,078 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:39,078 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:39,079 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:39,079 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:39,079 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:39,079 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:39,079 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:39,079 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:39,079 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:39,080 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:39,080 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:39,080 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:39,080 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:39,080 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:39,080 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:39,080 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:39,081 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:39,081 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:39,081 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:39,081 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:39,081 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:39,081 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:39,081 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:39,081 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:39,082 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:39,082 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:39,082 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:39,082 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:39,082 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:39,082 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:39,082 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:39,083 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:39,083 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,083 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:39,083 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,083 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,083 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,083 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:55:39,084 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:39,084 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:39,088 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076969088 2023-02-11 00:55:39,089 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,089 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:39,091 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:39,092 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:39,092 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:39,092 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076999092 2023-02-11 00:55:39,092 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077059092 2023-02-11 00:55:39,092 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:55:39,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN}] 2023-02-11 00:55:39,092 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:39,094 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:39,094 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:39,095 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076939094"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076939094"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076939094"}]},"ts":"1676076939094"} 2023-02-11 00:55:39,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:39,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,247 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3e2f92c6000f19c73dc861b8693fb931, disabling compactions & flushes 2023-02-11 00:55:39,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:39,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:39,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. after waiting 0 ms 2023-02-11 00:55:39,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:39,248 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 3e2f92c6000f19c73dc861b8693fb931 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 00:55:39,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/96f10b63478c48bd8550519e3b37ec12 2023-02-11 00:55:39,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/6b80f625f0194140b758351e56b800a3 2023-02-11 00:55:39,271 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf1/96f10b63478c48bd8550519e3b37ec12 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12 2023-02-11 00:55:39,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:55:39,274 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/.tmp/cf2/6b80f625f0194140b758351e56b800a3 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3 2023-02-11 00:55:39,277 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:55:39,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3e2f92c6000f19c73dc861b8693fb931 in 30ms, sequenceid=82, compaction requested=true 2023-02-11 00:55:39,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 00:55:39,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:39,283 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. 2023-02-11 00:55:39,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3e2f92c6000f19c73dc861b8693fb931: Waiting for close lock at 1676076939247Running coprocessor pre-close hooks at 1676076939247Disabling compacts and flushes for region at 1676076939247Disabling writes for close at 1676076939248 (+1 ms)Obtaining lock to block concurrent updates at 1676076939248Preparing flush snapshotting stores in 3e2f92c6000f19c73dc861b8693fb931 at 1676076939248Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676076939248Flushing stores of testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. at 1676076939248Flushing 3e2f92c6000f19c73dc861b8693fb931/cf1: creating writer at 1676076939249 (+1 ms)Flushing 3e2f92c6000f19c73dc861b8693fb931/cf1: appending metadata at 1676076939251 (+2 ms)Flushing 3e2f92c6000f19c73dc861b8693fb931/cf1: closing flushed file at 1676076939251Flushing 3e2f92c6000f19c73dc861b8693fb931/cf2: creating writer at 1676076939260 (+9 ms)Flushing 3e2f92c6000f19c73dc861b8693fb931/cf2: appending metadata at 1676076939262 (+2 ms)Flushing 3e2f92c6000f19c73dc861b8693fb931/cf2: closing flushed file at 1676076939262Flushing 3e2f92c6000f19c73dc861b8693fb931/cf1: reopening flushed file at 1676076939271 (+9 ms)Flushing 3e2f92c6000f19c73dc861b8693fb931/cf2: reopening flushed file at 1676076939275 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3e2f92c6000f19c73dc861b8693fb931 in 30ms, sequenceid=82, compaction requested=true at 1676076939278 (+3 ms)Writing region close event to WAL at 1676076939279 (+1 ms)Running coprocessor post-close hooks at 1676076939282 (+3 ms)Closed at 1676076939283 (+1 ms) 2023-02-11 00:55:39,285 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,285 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED 2023-02-11 00:55:39,285 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076939285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076939285"}]},"ts":"1676076939285"} 2023-02-11 00:55:39,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-11 00:55:39,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 in 190 msec 2023-02-11 00:55:39,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-11 00:55:39,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN in 196 msec 2023-02-11 00:55:39,291 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:39,291 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:39,291 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:39,342 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,342 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:39,342 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:39,342 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:39,342 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:39,343 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:39,343 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:39,343 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:39,343 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:39,343 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:39,344 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 00:55:39,344 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:55:39,345 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:55:39,345 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:39,345 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:39,345 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:39,345 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:39,345 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:39,346 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:39,346 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:39,346 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:39,346 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:39,346 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:39,346 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:39,346 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:39,346 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:39,347 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:39,347 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:39,347 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:39,347 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:39,347 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:39,347 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:39,347 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:39,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:39,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:39,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:39,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:39,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:39,348 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:39,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:39,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:39,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:39,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:39,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:39,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:39,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:39,349 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:39,350 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,350 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:39,350 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:39,350 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:39,350 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,350 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,350 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,350 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:39,350 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:39,350 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 00:55:39,351 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:39,351 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:39,352 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076969352 2023-02-11 00:55:39,352 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:39,353 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,355 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:39,355 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:39,355 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:39,355 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676076999355 2023-02-11 00:55:39,355 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077059355 2023-02-11 00:55:39,355 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:55:39,358 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:39,365 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=3e2f92c6000f19c73dc861b8693fb931, threads=14 2023-02-11 00:55:39,365 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,365 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,365 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,366 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,526 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,545 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,559 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,560 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,574 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,578 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,588 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,589 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,594 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,597 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,597 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,597 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,598 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,692 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 00:55:39,895 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:39,895 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 3e2f92c6000f19c73dc861b8693fb931 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:55:39,925 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:39,925 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:39,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:39,925 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:40,106 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,106 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:40,106 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:40,106 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:40,106 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:40,106 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:40,106 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:40,106 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:40,107 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:40,107 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:40,109 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 00:55:40,109 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:55:40,109 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:55:40,109 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:40,109 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:40,109 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:40,109 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:40,110 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,110 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:40,110 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,110 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,110 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,110 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:40,110 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,110 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,111 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,111 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:40,111 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:40,111 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,111 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,111 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,111 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:40,112 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,112 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,112 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,112 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:40,112 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,112 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,112 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,112 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,113 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:40,113 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,113 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,113 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,113 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,113 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,113 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:40,114 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,114 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:40,114 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:40,114 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:40,114 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,114 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,114 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,114 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:40,114 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:40,114 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:55:40,115 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:40,115 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:40,116 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076970116 2023-02-11 00:55:40,117 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:40,117 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,119 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:40,120 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:40,120 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:40,120 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676077000120 2023-02-11 00:55:40,120 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077060120 2023-02-11 00:55:40,120 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:55:40,127 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=3e2f92c6000f19c73dc861b8693fb931, threads=14 2023-02-11 00:55:40,128 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,128 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,128 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,128 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,128 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,128 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,129 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,129 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,129 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,129 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,131 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,131 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,132 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,132 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,213 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,219 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,222 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,222 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,224 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,225 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,230 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,232 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,232 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,232 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,232 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,236 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,237 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,237 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12 for region: 3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,237 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 3e2f92c6000f19c73dc861b8693fb931 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:55:40,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:40,269 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:55:40,270 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:55:40,270 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:40,270 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:40,270 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:40,370 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,370 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:40,370 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:40,370 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:40,370 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:40,371 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:40,371 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:40,371 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:40,371 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:40,371 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:40,372 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 00:55:40,372 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:55:40,372 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:55:40,373 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:40,373 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:40,373 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:40,373 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:40,373 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,373 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:40,374 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,374 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,374 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,374 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:40,374 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,374 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,374 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,374 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,375 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:40,376 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,376 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,376 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,376 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,376 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:40,376 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,376 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,376 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,377 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,377 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,377 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:40,377 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,377 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:40,377 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:40,377 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:40,377 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,377 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,377 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,377 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:40,378 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:40,378 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:55:40,378 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:40,379 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:40,379 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076970379 2023-02-11 00:55:40,379 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:40,381 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,383 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:40,383 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:40,383 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:40,383 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676077000383 2023-02-11 00:55:40,383 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077060383 2023-02-11 00:55:40,383 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:55:40,385 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:40,387 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:40,387 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:40,387 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:40,633 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,633 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:40,633 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:40,633 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:40,634 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:40,634 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:40,634 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:40,634 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:40,634 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:40,634 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:40,635 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 00:55:40,635 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:55:40,636 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:55:40,636 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:40,636 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:40,636 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:40,636 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:40,636 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,637 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:40,637 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,637 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,637 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,637 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:40,637 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,637 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,637 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,638 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:40,638 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:40,638 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,638 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,638 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,638 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:40,638 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,638 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,639 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,639 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:40,639 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,639 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,639 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,639 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,639 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:40,640 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:40,641 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:40,641 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,641 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,641 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,641 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:40,641 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:40,641 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:55:40,641 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:40,642 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:40,642 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076970642 2023-02-11 00:55:40,643 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:40,644 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,646 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:40,646 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:40,646 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:40,646 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676077000646 2023-02-11 00:55:40,646 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077060646 2023-02-11 00:55:40,646 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:55:40,646 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:40,648 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076940648"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676076940648"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676076940648"}]},"ts":"1676076940648"} 2023-02-11 00:55:40,648 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076940648"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076940648"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076940648"}]},"ts":"1676076940648"} 2023-02-11 00:55:40,648 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076940648"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076940648"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076940648"}]},"ts":"1676076940648"} 2023-02-11 00:55:40,650 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:40,650 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:40,650 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:40,896 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,896 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:40,897 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:40,897 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:40,897 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:40,897 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:40,897 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:40,897 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:40,897 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:40,898 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:40,899 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-11 00:55:40,899 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:55:40,899 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:55:40,900 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:40,900 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:40,900 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:40,901 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:40,901 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,901 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:40,901 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,901 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:40,902 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,902 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:40,902 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,902 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:40,903 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,903 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:40,904 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:40,904 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,904 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:40,904 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,904 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:40,904 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,905 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:40,905 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,905 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:40,905 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,905 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:40,906 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,906 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,907 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:40,907 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,907 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:40,907 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,907 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,907 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:40,908 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:40,908 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,908 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:40,908 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:40,908 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:40,909 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,909 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,909 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,909 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:40,909 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:40,909 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-11 00:55:40,910 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:40,910 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:40,911 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076970911 2023-02-11 00:55:40,913 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:40,914 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,915 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:40,915 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:40,916 INFO [Listener at localhost/35701] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,38317,1676076840434, table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:40,916 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=58a11ff02e7c55aa3f49c9b63866685b, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 00:55:40,916 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=36e140d2d42839db13899e8dd650df4d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 00:55:40,916 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:40,916 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676077000916 2023-02-11 00:55:40,916 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077060916 2023-02-11 00:55:40,916 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:55:40,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676076940917"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676076940917"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676076940917"}]},"ts":"1676076940917"} 2023-02-11 00:55:40,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076940917"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076940917"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076940917"}]},"ts":"1676076940917"} 2023-02-11 00:55:40,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076940917"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076940917"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076940917"}]},"ts":"1676076940917"} 2023-02-11 00:55:40,919 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:40,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:40,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:40,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:40,921 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:41,167 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,167 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:41,167 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:41,167 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:41,167 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:41,167 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:41,167 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:41,167 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:41,167 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:41,168 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:41,169 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-11 00:55:41,169 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:55:41,169 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:55:41,169 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:41,170 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:41,170 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:41,170 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:41,170 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,170 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:41,170 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,171 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,171 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,171 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:41,171 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,171 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,171 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,171 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,172 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:41,173 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,173 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,173 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,173 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,173 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:41,173 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,173 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,173 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,174 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,174 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,174 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:41,174 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,174 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:41,174 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,174 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:41,174 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,174 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,174 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,175 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,175 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,175 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:55:41,175 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:41,176 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:41,176 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076971176 2023-02-11 00:55:41,177 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:41,177 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,179 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:41,179 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:41,180 INFO [Listener at localhost/35701] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,38317,1676076840434, table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:41,180 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=58a11ff02e7c55aa3f49c9b63866685b, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 00:55:41,180 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=36e140d2d42839db13899e8dd650df4d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 00:55:41,180 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:41,180 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676077001180 2023-02-11 00:55:41,180 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077061180 2023-02-11 00:55:41,180 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 00:55:41,180 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:41,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, ASSIGN}] 2023-02-11 00:55:41,182 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:41,182 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:41,182 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:41,430 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,430 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:41,431 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:41,431 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:41,431 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:41,431 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:41,431 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:41,431 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:41,431 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:41,432 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 00:55:41,433 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-11 00:55:41,433 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:55:41,434 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:55:41,434 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:41,434 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:41,434 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:41,434 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:41,434 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,435 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:41,435 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,435 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,435 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,435 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:41,435 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,435 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,435 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,436 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:41,436 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:41,436 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,436 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,436 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,436 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:41,436 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,436 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,437 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,437 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:41,437 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,437 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,437 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,437 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,437 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:41,437 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,438 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:41,439 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,439 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,439 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,439 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,439 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,439 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 00:55:41,439 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:41,440 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:41,440 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076971440 2023-02-11 00:55:41,442 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:41,442 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,444 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:41,444 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:41,444 INFO [Listener at localhost/35701] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,38317,1676076840434, table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:41,444 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=58a11ff02e7c55aa3f49c9b63866685b, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 00:55:41,444 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=36e140d2d42839db13899e8dd650df4d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 00:55:41,444 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:41,444 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676077001444 2023-02-11 00:55:41,444 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077061444 2023-02-11 00:55:41,444 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 00:55:41,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, ASSIGN}] 2023-02-11 00:55:41,445 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:41,446 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, ASSIGN 2023-02-11 00:55:41,447 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:55:41,447 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, ASSIGN 2023-02-11 00:55:41,448 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,38317,1676076840434; forceNewPlan=false, retain=false 2023-02-11 00:55:41,597 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 00:55:41,599 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=36e140d2d42839db13899e8dd650df4d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,599 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076941599"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076941599"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076941599"}]},"ts":"1676076941599"} 2023-02-11 00:55:41,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 36e140d2d42839db13899e8dd650df4d, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:41,602 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=58a11ff02e7c55aa3f49c9b63866685b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,602 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076941602"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076941602"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076941602"}]},"ts":"1676076941602"} 2023-02-11 00:55:41,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 58a11ff02e7c55aa3f49c9b63866685b, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:41,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:41,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 36e140d2d42839db13899e8dd650df4d, NAME => 'testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 00:55:41,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:41,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:41,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,755 INFO [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,756 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1 2023-02-11 00:55:41,756 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1 2023-02-11 00:55:41,757 INFO [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36e140d2d42839db13899e8dd650df4d columnFamilyName cf1 2023-02-11 00:55:41,767 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d-top 2023-02-11 00:55:41,770 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2-top 2023-02-11 00:55:41,773 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6-top 2023-02-11 00:55:41,776 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c-top 2023-02-11 00:55:41,779 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a-top 2023-02-11 00:55:41,782 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55-top 2023-02-11 00:55:41,784 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12-top 2023-02-11 00:55:41,785 INFO [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(310): Store=36e140d2d42839db13899e8dd650df4d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:41,785 INFO [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,785 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2 2023-02-11 00:55:41,785 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2 2023-02-11 00:55:41,786 INFO [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 36e140d2d42839db13899e8dd650df4d columnFamilyName cf2 2023-02-11 00:55:41,798 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6-top 2023-02-11 00:55:41,801 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3-top 2023-02-11 00:55:41,804 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed-top 2023-02-11 00:55:41,807 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96-top 2023-02-11 00:55:41,810 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f-top 2023-02-11 00:55:41,813 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94-top 2023-02-11 00:55:41,816 DEBUG [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b-top 2023-02-11 00:55:41,816 INFO [StoreOpener-36e140d2d42839db13899e8dd650df4d-1] regionserver.HStore(310): Store=36e140d2d42839db13899e8dd650df4d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:41,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:41,820 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 36e140d2d42839db13899e8dd650df4d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59229194, jitterRate=-0.11741623282432556}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:41,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 36e140d2d42839db13899e8dd650df4d: Running coprocessor pre-open hook at 1676076941754Writing region info on filesystem at 1676076941754Initializing all the Stores at 1676076941755 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076941755Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076941755Cleaning up temporary data from old regions at 1676076941818 (+63 ms)Cleaning up detritus from prior splits at 1676076941819 (+1 ms)Running coprocessor post-open hooks at 1676076941821 (+2 ms)Region opened successfully at 1676076941821 2023-02-11 00:55:41,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d., pid=117, masterSystemTime=1676076941751 2023-02-11 00:55:41,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:41,824 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:41,824 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:41,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:55:41,824 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:41,824 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:41,824 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1912): 36e140d2d42839db13899e8dd650df4d/cf2 is initiating minor compaction (all files) 2023-02-11 00:55:41,824 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1912): 36e140d2d42839db13899e8dd650df4d/cf1 is initiating minor compaction (all files) 2023-02-11 00:55:41,825 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 36e140d2d42839db13899e8dd650df4d/cf2 in testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:41,825 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 36e140d2d42839db13899e8dd650df4d/cf1 in testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:41,825 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3-top] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/.tmp, totalSize=34.9 K 2023-02-11 00:55:41,825 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12-top] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/.tmp, totalSize=34.9 K 2023-02-11 00:55:41,826 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076937021 2023-02-11 00:55:41,826 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076937021 2023-02-11 00:55:41,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:41,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:41,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:41,826 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 58a11ff02e7c55aa3f49c9b63866685b, NAME => 'testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 00:55:41,826 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=36e140d2d42839db13899e8dd650df4d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:41,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076941826"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076941826"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076941826"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076941826"}]},"ts":"1676076941826"} 2023-02-11 00:55:41,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:41,827 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076937190 2023-02-11 00:55:41,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:41,827 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076937190 2023-02-11 00:55:41,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:41,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:41,828 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076937361 2023-02-11 00:55:41,828 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076937361 2023-02-11 00:55:41,828 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676076937528 2023-02-11 00:55:41,828 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676076937528 2023-02-11 00:55:41,828 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676076937692 2023-02-11 00:55:41,829 INFO [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:41,829 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676076937692 2023-02-11 00:55:41,829 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676076937859 2023-02-11 00:55:41,829 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676076937859 2023-02-11 00:55:41,830 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076938039 2023-02-11 00:55:41,830 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1 2023-02-11 00:55:41,830 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1 2023-02-11 00:55:41,830 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076938039 2023-02-11 00:55:41,830 INFO [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 58a11ff02e7c55aa3f49c9b63866685b columnFamilyName cf1 2023-02-11 00:55:41,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-11 00:55:41,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 36e140d2d42839db13899e8dd650df4d, server=jenkins-hbase4.apache.org,38317,1676076840434 in 228 msec 2023-02-11 00:55:41,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, ASSIGN in 386 msec 2023-02-11 00:55:41,850 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] throttle.PressureAwareThroughputController(145): 36e140d2d42839db13899e8dd650df4d#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:41,851 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] throttle.PressureAwareThroughputController(145): 36e140d2d42839db13899e8dd650df4d#cf2#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:41,851 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d-bottom 2023-02-11 00:55:41,855 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2-bottom 2023-02-11 00:55:41,857 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6-bottom 2023-02-11 00:55:41,861 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c-bottom 2023-02-11 00:55:41,864 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a-bottom 2023-02-11 00:55:41,872 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55-bottom 2023-02-11 00:55:41,877 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/.tmp/cf1/21597c2355f74238b15ea37df5c294a1 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/21597c2355f74238b15ea37df5c294a1 2023-02-11 00:55:41,877 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12-bottom 2023-02-11 00:55:41,877 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/.tmp/cf2/db67f8cc2fcd4d4da5e9250575838b09 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/db67f8cc2fcd4d4da5e9250575838b09 2023-02-11 00:55:41,877 INFO [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(310): Store=58a11ff02e7c55aa3f49c9b63866685b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:41,877 INFO [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:41,878 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2 2023-02-11 00:55:41,878 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2 2023-02-11 00:55:41,879 INFO [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 58a11ff02e7c55aa3f49c9b63866685b columnFamilyName cf2 2023-02-11 00:55:41,883 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 36e140d2d42839db13899e8dd650df4d/cf2 of 36e140d2d42839db13899e8dd650df4d into db67f8cc2fcd4d4da5e9250575838b09(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:41,883 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 36e140d2d42839db13899e8dd650df4d: 2023-02-11 00:55:41,884 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d., storeName=36e140d2d42839db13899e8dd650df4d/cf2, priority=9, startTime=1676076941824; duration=0sec 2023-02-11 00:55:41,884 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:41,884 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 36e140d2d42839db13899e8dd650df4d/cf1 of 36e140d2d42839db13899e8dd650df4d into 21597c2355f74238b15ea37df5c294a1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:41,884 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 36e140d2d42839db13899e8dd650df4d: 2023-02-11 00:55:41,884 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d., storeName=36e140d2d42839db13899e8dd650df4d/cf1, priority=9, startTime=1676076941821; duration=0sec 2023-02-11 00:55:41,885 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:41,894 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6-bottom 2023-02-11 00:55:41,897 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3-bottom 2023-02-11 00:55:41,899 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed-bottom 2023-02-11 00:55:41,902 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96-bottom 2023-02-11 00:55:41,905 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f-bottom 2023-02-11 00:55:41,908 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94-bottom 2023-02-11 00:55:41,911 DEBUG [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b-bottom 2023-02-11 00:55:41,911 INFO [StoreOpener-58a11ff02e7c55aa3f49c9b63866685b-1] regionserver.HStore(310): Store=58a11ff02e7c55aa3f49c9b63866685b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:41,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:41,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:41,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:41,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:41,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 58a11ff02e7c55aa3f49c9b63866685b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61081928, jitterRate=-0.08980834484100342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:41,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 58a11ff02e7c55aa3f49c9b63866685b: Running coprocessor pre-open hook at 1676076941827Writing region info on filesystem at 1676076941827Initializing all the Stores at 1676076941828 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076941828Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076941829 (+1 ms)Cleaning up temporary data from old regions at 1676076941913 (+84 ms)Cleaning up detritus from prior splits at 1676076941913Running coprocessor post-open hooks at 1676076941915 (+2 ms)Region opened successfully at 1676076941915 2023-02-11 00:55:41,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b., pid=118, masterSystemTime=1676076941751 2023-02-11 00:55:41,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:41,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:41,917 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:41,917 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:41,917 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:41,917 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1912): 58a11ff02e7c55aa3f49c9b63866685b/cf2 is initiating minor compaction (all files) 2023-02-11 00:55:41,917 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:41,917 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 58a11ff02e7c55aa3f49c9b63866685b/cf2 in testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:41,917 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1912): 58a11ff02e7c55aa3f49c9b63866685b/cf1 is initiating minor compaction (all files) 2023-02-11 00:55:41,917 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 58a11ff02e7c55aa3f49c9b63866685b/cf1 in testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:41,917 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/.tmp, totalSize=34.9 K 2023-02-11 00:55:41,917 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/.tmp, totalSize=34.9 K 2023-02-11 00:55:41,918 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076937021 2023-02-11 00:55:41,918 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076937021 2023-02-11 00:55:41,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:41,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:41,918 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076937190 2023-02-11 00:55:41,918 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076937190 2023-02-11 00:55:41,919 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=58a11ff02e7c55aa3f49c9b63866685b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,919 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076937361 2023-02-11 00:55:41,919 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076937361 2023-02-11 00:55:41,919 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076941919"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076941919"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076941919"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076941919"}]},"ts":"1676076941919"} 2023-02-11 00:55:41,919 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076937528 2023-02-11 00:55:41,919 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076937528 2023-02-11 00:55:41,920 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076937692 2023-02-11 00:55:41,920 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076937692 2023-02-11 00:55:41,920 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076937859 2023-02-11 00:55:41,920 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076937859 2023-02-11 00:55:41,920 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] compactions.Compactor(207): Compacting 6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076938039 2023-02-11 00:55:41,921 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] compactions.Compactor(207): Compacting 96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076938039 2023-02-11 00:55:41,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-11 00:55:41,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 58a11ff02e7c55aa3f49c9b63866685b, server=jenkins-hbase4.apache.org,38317,1676076840434 in 317 msec 2023-02-11 00:55:41,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-11 00:55:41,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, ASSIGN in 478 msec 2023-02-11 00:55:41,925 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 00:55:41,925 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:41,925 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 00:55:41,930 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] throttle.PressureAwareThroughputController(145): 58a11ff02e7c55aa3f49c9b63866685b#cf2#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:41,930 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] throttle.PressureAwareThroughputController(145): 58a11ff02e7c55aa3f49c9b63866685b#cf1#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:41,945 INFO [Listener at localhost/35701] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,945 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 00:55:41,945 INFO [Listener at localhost/35701] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:41,945 INFO [Listener at localhost/35701] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:41,945 ERROR [Listener at localhost/35701] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 00:55:41,945 INFO [Listener at localhost/35701] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 00:55:41,945 INFO [Listener at localhost/35701] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 00:55:41,946 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 00:55:41,946 INFO [Listener at localhost/35701] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 00:55:41,951 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 5 msec 2023-02-11 00:55:41,953 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-11 00:55:41,953 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 00:55:41,953 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 00:55:41,953 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-11 00:55:41,953 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-11 00:55:41,953 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-11 00:55:41,954 DEBUG [Listener at localhost/35701] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-11 00:55:41,954 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 00:55:41,954 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 00:55:41,954 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 00:55:41,955 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 00:55:41,955 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,955 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b0843d8ffc44a9678fe6589a79fd74e7, daughterA=386584ef8660be854e06d144066a6e91, daughterB=f6372215e78dd6cdebad6e3e039951b1 2023-02-11 00:55:41,955 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,955 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 00:55:41,956 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,956 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=a4578dcd4830de76f0230a8be7356613, daughterA=536c8203ff25abc271313ce8e0f84cc7, daughterB=129c0662a2141871c7ffd1b7a430c9e9 2023-02-11 00:55:41,956 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,956 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 00:55:41,957 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,957 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=517bceec69b190e0374f76671f87d5d1, daughterA=ef64b99a250a430b37eb8cd170d91141, daughterB=b07e9e94539895eecfacfe698f4f45f1 2023-02-11 00:55:41,957 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=517bceec69b190e0374f76671f87d5d1, ASSIGN 2023-02-11 00:55:41,957 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,957 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 00:55:41,958 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,958 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=2645d4c46317f3690a043a36a348b9c1, daughterA=95f542afca1500322e0c0b9924479821, daughterB=639a8ff09cb3f0dc787eb4be08877b2e 2023-02-11 00:55:41,958 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,958 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 00:55:41,958 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,959 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e3beb69538b330eb0b73628e03f86d35, daughterA=3672dd666d16ab8bcdec33d37c2608f2, daughterB=6f03a7215e0b89c529706aaf04e65832 2023-02-11 00:55:41,959 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,959 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 00:55:41,959 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,959 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41445,1676076840467, table=testRollbackForSplitTableRegionWithReplica, region=6d7673c1d5364ef92bcb8dca986b97c6 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f114625497f027c6d8b23a22cc933405, daughterA=761e7de882914d462e2467abc215e26b, daughterB=8c620c4534f5aa932f7ff956d6b553db 2023-02-11 00:55:41,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 00:55:41,960 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 00:55:41,961 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:41,962 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/.tmp/cf2/a7b88d39dc6a405fb1e6b20afe819dfe as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/a7b88d39dc6a405fb1e6b20afe819dfe 2023-02-11 00:55:41,962 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,962 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/.tmp/cf1/68c3aaa5486148278207e05c2803d907 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/68c3aaa5486148278207e05c2803d907 2023-02-11 00:55:41,962 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN 2023-02-11 00:55:41,963 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,963 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, ASSIGN 2023-02-11 00:55:41,963 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, ASSIGN 2023-02-11 00:55:41,963 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 36e140d2d42839db13899e8dd650df4d, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,963 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 58a11ff02e7c55aa3f49c9b63866685b, server=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:41,963 DEBUG [Listener at localhost/35701] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d held the lock before restarting, call acquireLock to restore it. 2023-02-11 00:55:41,963 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,964 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,964 INFO [Listener at localhost/35701] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,964 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,964 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 3e2f92c6000f19c73dc861b8693fb931, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,964 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,964 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,964 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 36e140d2d42839db13899e8dd650df4d, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,964 DEBUG [Listener at localhost/35701] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 58a11ff02e7c55aa3f49c9b63866685b, server=jenkins-hbase4.apache.org,38317,1676076840434 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 00:55:41,964 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-11 00:55:41,965 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 00:55:41,966 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386, table=hbase:meta, region=1588230740 2023-02-11 00:55:41,967 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 58a11ff02e7c55aa3f49c9b63866685b/cf2 of 58a11ff02e7c55aa3f49c9b63866685b into a7b88d39dc6a405fb1e6b20afe819dfe(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:41,967 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 58a11ff02e7c55aa3f49c9b63866685b: 2023-02-11 00:55:41,967 INFO [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b., storeName=58a11ff02e7c55aa3f49c9b63866685b/cf2, priority=9, startTime=1676076941917; duration=0sec 2023-02-11 00:55:41,967 DEBUG [RS:1;jenkins-hbase4:38317-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:41,971 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676076971971 2023-02-11 00:55:41,973 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:41,973 DEBUG [Listener at localhost/35701] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 00:55:41,974 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 58a11ff02e7c55aa3f49c9b63866685b/cf1 of 58a11ff02e7c55aa3f49c9b63866685b into 68c3aaa5486148278207e05c2803d907(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:41,974 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 58a11ff02e7c55aa3f49c9b63866685b: 2023-02-11 00:55:41,974 INFO [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b., storeName=58a11ff02e7c55aa3f49c9b63866685b/cf1, priority=9, startTime=1676076941916; duration=0sec 2023-02-11 00:55:41,974 DEBUG [RS:1;jenkins-hbase4:38317-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:41,976 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=b9396c6e19f6c2ad076604e11fcfef4f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:41,976 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=3e2f92c6000f19c73dc861b8693fb931, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=2 2023-02-11 00:55:41,977 INFO [Listener at localhost/35701] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,38317,1676076840434, table=testRecoveryAndDoubleExecution, region=3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:41,977 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=58a11ff02e7c55aa3f49c9b63866685b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=86 2023-02-11 00:55:41,977 DEBUG [Listener at localhost/35701] assignment.RegionStateStore(147): Load hbase:meta entry region=36e140d2d42839db13899e8dd650df4d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,38317,1676076840434, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434, openSeqNum=86 2023-02-11 00:55:41,977 INFO [Listener at localhost/35701] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 00:55:41,977 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676077001977 2023-02-11 00:55:41,977 INFO [Listener at localhost/35701] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676077061977 2023-02-11 00:55:41,977 INFO [Listener at localhost/35701] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 00:55:41,978 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 00:55:41,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=3e2f92c6000f19c73dc861b8693fb931, daughterA=58a11ff02e7c55aa3f49c9b63866685b, daughterB=36e140d2d42839db13899e8dd650df4d in 3.6810 sec 2023-02-11 00:55:42,187 ERROR [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-74375045-172.31.14.131-1676076836114 (Datanode Uuid e08011bb-ec6b-4682-9527-eafae3012242) service to localhost/127.0.0.1:46775 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 00:55:42,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38317] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:45472 deadline: 1676077002228, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931. is not online on jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:42,341 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 00:55:42,345 INFO [Listener at localhost/35701] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 00:55:42,347 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 00:55:42,355 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:42,356 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-11 00:55:42,357 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-11 00:55:42,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:42,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 00:55:42,359 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076942359"}]},"ts":"1676076942359"} 2023-02-11 00:55:42,360 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-11 00:55:42,362 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-11 00:55:42,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, UNASSIGN}] 2023-02-11 00:55:42,363 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, UNASSIGN 2023-02-11 00:55:42,363 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=36e140d2d42839db13899e8dd650df4d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:42,363 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076942363"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076942363"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076942363"}]},"ts":"1676076942363"} 2023-02-11 00:55:42,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 36e140d2d42839db13899e8dd650df4d, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:42,366 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, UNASSIGN 2023-02-11 00:55:42,366 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=58a11ff02e7c55aa3f49c9b63866685b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:42,366 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076942366"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076942366"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076942366"}]},"ts":"1676076942366"} 2023-02-11 00:55:42,367 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 58a11ff02e7c55aa3f49c9b63866685b, server=jenkins-hbase4.apache.org,38317,1676076840434}] 2023-02-11 00:55:42,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 00:55:42,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:42,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 36e140d2d42839db13899e8dd650df4d, disabling compactions & flushes 2023-02-11 00:55:42,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:42,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:42,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. after waiting 0 ms 2023-02-11 00:55:42,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:42,517 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12-top] to archive 2023-02-11 00:55:42,518 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:42,519 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,520 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,521 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,522 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,523 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,523 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,525 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,526 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3-top] to archive 2023-02-11 00:55:42,526 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:42,527 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,528 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,529 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,530 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,531 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,532 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,533 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:55:42,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:42,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d. 2023-02-11 00:55:42,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 36e140d2d42839db13899e8dd650df4d: Waiting for close lock at 1676076942516Running coprocessor pre-close hooks at 1676076942516Disabling compacts and flushes for region at 1676076942516Disabling writes for close at 1676076942516Writing region close event to WAL at 1676076942534 (+18 ms)Running coprocessor post-close hooks at 1676076942536 (+2 ms)Closed at 1676076942536 2023-02-11 00:55:42,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:42,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:42,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 58a11ff02e7c55aa3f49c9b63866685b, disabling compactions & flushes 2023-02-11 00:55:42,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:42,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:42,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. after waiting 0 ms 2023-02-11 00:55:42,539 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:42,539 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=36e140d2d42839db13899e8dd650df4d, regionState=CLOSED 2023-02-11 00:55:42,539 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076942539"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076942539"}]},"ts":"1676076942539"} 2023-02-11 00:55:42,539 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12-bottom] to archive 2023-02-11 00:55:42,540 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:42,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-11 00:55:42,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 36e140d2d42839db13899e8dd650df4d, server=jenkins-hbase4.apache.org,38317,1676076840434 in 176 msec 2023-02-11 00:55:42,542 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=36e140d2d42839db13899e8dd650df4d, UNASSIGN in 180 msec 2023-02-11 00:55:42,543 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,544 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,545 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,546 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,547 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,547 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,549 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3-bottom] to archive 2023-02-11 00:55:42,549 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:42,551 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,551 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,552 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,553 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,554 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,555 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,555 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:55:42,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:42,559 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b. 2023-02-11 00:55:42,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 58a11ff02e7c55aa3f49c9b63866685b: Waiting for close lock at 1676076942538Running coprocessor pre-close hooks at 1676076942538Disabling compacts and flushes for region at 1676076942538Disabling writes for close at 1676076942539 (+1 ms)Writing region close event to WAL at 1676076942556 (+17 ms)Running coprocessor post-close hooks at 1676076942558 (+2 ms)Closed at 1676076942559 (+1 ms) 2023-02-11 00:55:42,560 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:42,561 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=58a11ff02e7c55aa3f49c9b63866685b, regionState=CLOSED 2023-02-11 00:55:42,561 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676076942561"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076942561"}]},"ts":"1676076942561"} 2023-02-11 00:55:42,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-11 00:55:42,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 58a11ff02e7c55aa3f49c9b63866685b, server=jenkins-hbase4.apache.org,38317,1676076840434 in 195 msec 2023-02-11 00:55:42,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-11 00:55:42,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=58a11ff02e7c55aa3f49c9b63866685b, UNASSIGN in 201 msec 2023-02-11 00:55:42,565 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076942565"}]},"ts":"1676076942565"} 2023-02-11 00:55:42,566 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-11 00:55:42,567 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-11 00:55:42,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 211 msec 2023-02-11 00:55:42,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 00:55:42,661 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-11 00:55:42,661 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-11 00:55:42,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:42,667 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:42,668 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:42,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-11 00:55:42,672 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,672 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:42,672 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:42,675 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/recovered.edits] 2023-02-11 00:55:42,676 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/recovered.edits] 2023-02-11 00:55:42,677 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/recovered.edits] 2023-02-11 00:55:42,681 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/1576d7857a5a4b60b2ec493cee34b75d 2023-02-11 00:55:42,681 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/.regioninfo to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/.regioninfo 2023-02-11 00:55:42,683 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/22d1f5e7d3864399a120dfc24ba538f2 2023-02-11 00:55:42,684 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/.regioninfo to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/.regioninfo 2023-02-11 00:55:42,685 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6 2023-02-11 00:55:42,686 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,686 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/55f52ffbe55444279268ca8905cbf75c 2023-02-11 00:55:42,688 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/1576d7857a5a4b60b2ec493cee34b75d.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,692 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/56839b6a9ad2464380c7ffc75ec8833a 2023-02-11 00:55:42,692 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,694 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/22d1f5e7d3864399a120dfc24ba538f2.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,694 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/5dded02a93114e76b118bdd920b6ce55 2023-02-11 00:55:42,694 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,695 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/32cf1b3d1f9e4d12bc17fdce28a961d6.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,696 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf1/96f10b63478c48bd8550519e3b37ec12 2023-02-11 00:55:42,696 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,697 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/55f52ffbe55444279268ca8905cbf75c.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,698 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,698 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/56839b6a9ad2464380c7ffc75ec8833a.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,700 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/55faf55c0afd44c1afa42756d93ec3e6 2023-02-11 00:55:42,700 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,700 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/5dded02a93114e76b118bdd920b6ce55.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,701 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,701 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/6b80f625f0194140b758351e56b800a3 2023-02-11 00:55:42,702 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf1/96f10b63478c48bd8550519e3b37ec12.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,703 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/8d779e79780842cca528df4a4fbbe4ed 2023-02-11 00:55:42,705 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/93485d29aa1449349e16411ec9417e96 2023-02-11 00:55:42,705 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,705 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/55faf55c0afd44c1afa42756d93ec3e6.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,706 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/b2860b675b374dffb24da0aae58b8a0f 2023-02-11 00:55:42,706 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,706 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/6b80f625f0194140b758351e56b800a3.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,707 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/bc3d93b722b241959473c39baf754e94 2023-02-11 00:55:42,707 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,708 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/8d779e79780842cca528df4a4fbbe4ed.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,709 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/cf2/c37556de43e7405e923aa0a0b43c569b 2023-02-11 00:55:42,709 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,709 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/93485d29aa1449349e16411ec9417e96.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,711 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,711 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/b2860b675b374dffb24da0aae58b8a0f.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,712 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,712 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/bc3d93b722b241959473c39baf754e94.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,712 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/recovered.edits/85.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931/recovered.edits/85.seqid 2023-02-11 00:55:42,713 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,714 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/36e140d2d42839db13899e8dd650df4d/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,714 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/58a11ff02e7c55aa3f49c9b63866685b/cf2/c37556de43e7405e923aa0a0b43c569b.3e2f92c6000f19c73dc861b8693fb931 2023-02-11 00:55:42,716 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/21597c2355f74238b15ea37df5c294a1 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf1/21597c2355f74238b15ea37df5c294a1 2023-02-11 00:55:42,716 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/68c3aaa5486148278207e05c2803d907 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf1/68c3aaa5486148278207e05c2803d907 2023-02-11 00:55:42,718 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/db67f8cc2fcd4d4da5e9250575838b09 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/cf2/db67f8cc2fcd4d4da5e9250575838b09 2023-02-11 00:55:42,718 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/a7b88d39dc6a405fb1e6b20afe819dfe to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/cf2/a7b88d39dc6a405fb1e6b20afe819dfe 2023-02-11 00:55:42,721 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d/recovered.edits/90.seqid 2023-02-11 00:55:42,721 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b/recovered.edits/90.seqid 2023-02-11 00:55:42,722 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/36e140d2d42839db13899e8dd650df4d 2023-02-11 00:55:42,722 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testRecoveryAndDoubleExecution/58a11ff02e7c55aa3f49c9b63866685b 2023-02-11 00:55:42,722 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-11 00:55:42,724 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:42,726 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-11 00:55:42,727 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-11 00:55:42,728 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:42,728 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-11 00:55:42,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076942728"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:42,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076942728"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:42,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076942728"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:42,730 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 00:55:42,730 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3e2f92c6000f19c73dc861b8693fb931, NAME => 'testRecoveryAndDoubleExecution,,1676076936518.3e2f92c6000f19c73dc861b8693fb931.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 58a11ff02e7c55aa3f49c9b63866685b, NAME => 'testRecoveryAndDoubleExecution,,1676076938297.58a11ff02e7c55aa3f49c9b63866685b.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 36e140d2d42839db13899e8dd650df4d, NAME => 'testRecoveryAndDoubleExecution,41,1676076938297.36e140d2d42839db13899e8dd650df4d.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 00:55:42,730 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-11 00:55:42,730 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076942730"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:42,732 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-11 00:55:42,733 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 00:55:42,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 71 msec 2023-02-11 00:55:42,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-11 00:55:42,769 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-11 00:55:42,783 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=373 (was 366) - Thread LEAK? -, OpenFileDescriptor=831 (was 745) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=103 (was 113), ProcessCount=168 (was 168), AvailableMemoryMB=2941 (was 2969) 2023-02-11 00:55:42,799 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=373, OpenFileDescriptor=831, MaxFileDescriptor=60000, SystemLoadAverage=103, ProcessCount=168, AvailableMemoryMB=2941 2023-02-11 00:55:42,800 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:42,803 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:55:42,803 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:55:42,804 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 00:55:42,806 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:55:42,806 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:55:42,808 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:42,808 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8 empty. 2023-02-11 00:55:42,808 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:42,808 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-11 00:55:42,831 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 00:55:42,832 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 154356c6ea2dcb7760e343c32daf69e8, NAME => 'testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:55:42,850 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:42,850 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 154356c6ea2dcb7760e343c32daf69e8, disabling compactions & flushes 2023-02-11 00:55:42,850 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:42,850 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:42,851 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. after waiting 0 ms 2023-02-11 00:55:42,851 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:42,851 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:42,851 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 154356c6ea2dcb7760e343c32daf69e8: Waiting for close lock at 1676076942850Disabling compacts and flushes for region at 1676076942850Disabling writes for close at 1676076942851 (+1 ms)Writing region close event to WAL at 1676076942851Closed at 1676076942851 2023-02-11 00:55:42,853 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:55:42,853 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076942853"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076942853"}]},"ts":"1676076942853"} 2023-02-11 00:55:42,855 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:55:42,855 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:55:42,856 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076942855"}]},"ts":"1676076942855"} 2023-02-11 00:55:42,857 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-11 00:55:42,860 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:55:42,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:55:42,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:55:42,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:55:42,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:55:42,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=154356c6ea2dcb7760e343c32daf69e8, ASSIGN}] 2023-02-11 00:55:42,863 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=154356c6ea2dcb7760e343c32daf69e8, ASSIGN 2023-02-11 00:55:42,864 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=154356c6ea2dcb7760e343c32daf69e8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41445,1676076840467; forceNewPlan=false, retain=false 2023-02-11 00:55:43,014 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:55:43,015 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=154356c6ea2dcb7760e343c32daf69e8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,015 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076943015"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076943015"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076943015"}]},"ts":"1676076943015"} 2023-02-11 00:55:43,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 154356c6ea2dcb7760e343c32daf69e8, server=jenkins-hbase4.apache.org,41445,1676076840467}] 2023-02-11 00:55:43,107 ERROR [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-74375045-172.31.14.131-1676076836114 (Datanode Uuid 0b950d92-fb4b-4a88-add0-dd4dcb24d701) service to localhost/127.0.0.1:46775 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 00:55:43,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:43,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 154356c6ea2dcb7760e343c32daf69e8, NAME => 'testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:55:43,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:43,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:43,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:43,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:43,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:43,174 INFO [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:43,175 DEBUG [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1 2023-02-11 00:55:43,175 DEBUG [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1 2023-02-11 00:55:43,176 INFO [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 154356c6ea2dcb7760e343c32daf69e8 columnFamilyName cf1 2023-02-11 00:55:43,176 INFO [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] regionserver.HStore(310): Store=154356c6ea2dcb7760e343c32daf69e8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:43,176 INFO [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:43,177 DEBUG [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2 2023-02-11 00:55:43,177 DEBUG [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2 2023-02-11 00:55:43,177 INFO [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 154356c6ea2dcb7760e343c32daf69e8 columnFamilyName cf2 2023-02-11 00:55:43,178 INFO [StoreOpener-154356c6ea2dcb7760e343c32daf69e8-1] regionserver.HStore(310): Store=154356c6ea2dcb7760e343c32daf69e8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:43,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:43,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:43,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:43,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:43,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:55:43,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 154356c6ea2dcb7760e343c32daf69e8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68321158, jitterRate=0.01806458830833435}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:43,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 154356c6ea2dcb7760e343c32daf69e8: Running coprocessor pre-open hook at 1676076943173Writing region info on filesystem at 1676076943173Initializing all the Stores at 1676076943174 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076943174Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076943174Cleaning up temporary data from old regions at 1676076943179 (+5 ms)Cleaning up detritus from prior splits at 1676076943179Running coprocessor post-open hooks at 1676076943182 (+3 ms)Region opened successfully at 1676076943182 2023-02-11 00:55:43,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8., pid=127, masterSystemTime=1676076943168 2023-02-11 00:55:43,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:43,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:43,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=154356c6ea2dcb7760e343c32daf69e8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076943184"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076943184"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076943184"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076943184"}]},"ts":"1676076943184"} 2023-02-11 00:55:43,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-11 00:55:43,187 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 154356c6ea2dcb7760e343c32daf69e8, server=jenkins-hbase4.apache.org,41445,1676076840467 in 169 msec 2023-02-11 00:55:43,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-11 00:55:43,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=154356c6ea2dcb7760e343c32daf69e8, ASSIGN in 325 msec 2023-02-11 00:55:43,189 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:55:43,189 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076943189"}]},"ts":"1676076943189"} 2023-02-11 00:55:43,190 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-11 00:55:43,192 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:55:43,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 389 msec 2023-02-11 00:55:43,307 DEBUG [Listener at localhost/35701] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 00:55:43,310 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44478, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 00:55:43,316 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:43,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,317 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:43,317 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:43,317 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 00:55:43,317 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 00:55:43,317 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,317 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,319 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,319 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,319 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,319 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,320 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,320 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:43,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:43,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:43,321 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:43,322 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:43,322 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:43,322 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:43,322 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:43,322 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:43,322 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:43,322 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. started... 2023-02-11 00:55:43,322 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 154356c6ea2dcb7760e343c32daf69e8 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 00:55:43,323 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,323 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,323 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:43,324 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,324 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,324 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:43,731 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/5080b0776d74409e8d3f182c522f1fac 2023-02-11 00:55:43,749 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/2c29c4786e964973a476f7f4815fffb4 2023-02-11 00:55:43,752 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/5080b0776d74409e8d3f182c522f1fac as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/5080b0776d74409e8d3f182c522f1fac 2023-02-11 00:55:43,756 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/5080b0776d74409e8d3f182c522f1fac, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:43,757 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/2c29c4786e964973a476f7f4815fffb4 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/2c29c4786e964973a476f7f4815fffb4 2023-02-11 00:55:43,760 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/2c29c4786e964973a476f7f4815fffb4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:43,761 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 154356c6ea2dcb7760e343c32daf69e8 in 439ms, sequenceid=6, compaction requested=false 2023-02-11 00:55:43,761 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:43,761 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 154356c6ea2dcb7760e343c32daf69e8: 2023-02-11 00:55:43,762 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:43,762 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:43,762 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:43,762 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,762 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:43,762 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:43,763 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,763 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:43,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:43,764 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,764 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:43,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:43,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:43,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:43,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 00:55:43,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@720ae183[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:43,766 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 00:55:43,766 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:43,768 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:43,768 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,768 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:43,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,768 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:43,768 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:43,769 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:43,770 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:43,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:43,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:43,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:43,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:43,771 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:43,771 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:43,771 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:43,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:43,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:43,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41445,1676076840467': 2023-02-11 00:55:43,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:55:43,774 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 00:55:43,774 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:43,774 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:43,774 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,774 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:43,775 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,775 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,775 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,775 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,775 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,775 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:43,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,776 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,776 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,776 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,777 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,778 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:43,778 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,779 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,789 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,789 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,789 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,789 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,789 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,790 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:43,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,790 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:43,790 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,790 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,790 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,790 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,790 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:43,790 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,790 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,791 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,791 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:43,791 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,791 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:43,791 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,791 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,791 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,791 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,891 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:43,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:43,902 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:43,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:43,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,903 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:43,903 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:43,903 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 00:55:43,903 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 00:55:43,903 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,903 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,904 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,904 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,904 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,904 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,905 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,905 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:43,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,905 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,905 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:43,906 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:43,906 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,906 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:43,906 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:43,907 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:43,907 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:43,907 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. started... 2023-02-11 00:55:43,907 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 154356c6ea2dcb7760e343c32daf69e8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:43,908 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,909 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,909 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:43,909 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,909 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,909 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:43,915 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/ab4cb2200c634d93b6982048f726b48f 2023-02-11 00:55:43,924 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/d9b50a36871d4122b0d32eafe4925241 2023-02-11 00:55:43,927 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/ab4cb2200c634d93b6982048f726b48f as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/ab4cb2200c634d93b6982048f726b48f 2023-02-11 00:55:43,930 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/ab4cb2200c634d93b6982048f726b48f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:55:43,930 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/d9b50a36871d4122b0d32eafe4925241 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/d9b50a36871d4122b0d32eafe4925241 2023-02-11 00:55:43,933 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/d9b50a36871d4122b0d32eafe4925241, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:55:43,934 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 154356c6ea2dcb7760e343c32daf69e8 in 27ms, sequenceid=19, compaction requested=false 2023-02-11 00:55:43,934 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 154356c6ea2dcb7760e343c32daf69e8: 2023-02-11 00:55:43,934 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:43,934 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:43,934 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:43,934 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,934 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:43,934 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:43,936 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,936 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:43,937 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,937 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:43,938 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 00:55:43,938 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e0380f8[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:43,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 00:55:43,938 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,939 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,939 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,939 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:43,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:43,940 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,940 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:43,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:43,940 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:43,941 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:43,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:43,941 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:43,942 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:43,942 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:43,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:43,942 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:43,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:43,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:43,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41445,1676076840467': 2023-02-11 00:55:43,944 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:55:43,944 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 00:55:43,945 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:43,945 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:43,945 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,945 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:43,947 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,947 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,947 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,947 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,947 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,947 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,947 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:43,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,948 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:43,948 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,949 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,950 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:43,950 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,950 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,953 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,953 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,953 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,953 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,953 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:43,953 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:43,953 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:43,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,953 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,953 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:43,953 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:43,953 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:43,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:43,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:43,954 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:43,955 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:43,955 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:43,955 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,054 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:44,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:44,064 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:44,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:44,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,066 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:44,066 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,066 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 00:55:44,066 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 00:55:44,066 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,066 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,067 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,067 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,067 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,067 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,068 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,068 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,069 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,069 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:44,070 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. started... 2023-02-11 00:55:44,070 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 154356c6ea2dcb7760e343c32daf69e8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,478 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/de066b44c1764306bee82d61a40154df 2023-02-11 00:55:44,488 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/38a019d9dcc447f4b48744ed8a7696e0 2023-02-11 00:55:44,491 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/de066b44c1764306bee82d61a40154df as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/de066b44c1764306bee82d61a40154df 2023-02-11 00:55:44,494 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/de066b44c1764306bee82d61a40154df, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:55:44,495 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/38a019d9dcc447f4b48744ed8a7696e0 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/38a019d9dcc447f4b48744ed8a7696e0 2023-02-11 00:55:44,498 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/38a019d9dcc447f4b48744ed8a7696e0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:55:44,499 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 154356c6ea2dcb7760e343c32daf69e8 in 429ms, sequenceid=32, compaction requested=true 2023-02-11 00:55:44,499 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 154356c6ea2dcb7760e343c32daf69e8: 2023-02-11 00:55:44,499 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:44,499 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:44,499 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:44,499 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,499 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,499 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,500 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,500 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,501 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,501 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 00:55:44,502 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d465e56[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:44,502 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 00:55:44,503 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,503 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,503 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,503 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,504 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:44,504 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:44,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,504 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:44,509 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,509 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,509 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,510 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41445,1676076840467': 2023-02-11 00:55:44,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:55:44,513 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 00:55:44,513 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:44,513 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:44,513 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,513 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:44,514 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,514 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,514 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,514 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,514 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,514 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,515 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,514 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,515 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,516 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,517 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,517 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,517 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,519 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,520 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:44,520 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,520 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,520 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,521 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,521 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,521 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,521 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,521 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,620 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:44,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:44,635 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:44,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:44,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,637 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:44,637 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,637 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 00:55:44,637 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 00:55:44,637 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,637 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,639 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,640 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,639 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,640 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,640 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,640 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:44,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,642 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,642 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,643 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,643 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:44,643 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. started... 2023-02-11 00:55:44,643 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 154356c6ea2dcb7760e343c32daf69e8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,644 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,645 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,655 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/acaec320e3e34a90a5a7666c37739fc7 2023-02-11 00:55:44,665 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/0785c8f104de442d89d8df43e94e8452 2023-02-11 00:55:44,668 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/acaec320e3e34a90a5a7666c37739fc7 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/acaec320e3e34a90a5a7666c37739fc7 2023-02-11 00:55:44,671 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/acaec320e3e34a90a5a7666c37739fc7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:55:44,672 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/0785c8f104de442d89d8df43e94e8452 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/0785c8f104de442d89d8df43e94e8452 2023-02-11 00:55:44,675 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/0785c8f104de442d89d8df43e94e8452, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:55:44,675 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 154356c6ea2dcb7760e343c32daf69e8 in 32ms, sequenceid=45, compaction requested=true 2023-02-11 00:55:44,675 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 154356c6ea2dcb7760e343c32daf69e8: 2023-02-11 00:55:44,675 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:44,675 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:44,676 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:44,676 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,676 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,676 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,677 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,677 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 00:55:44,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@351a7a61[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:44,679 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 00:55:44,679 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,680 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,681 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:44,680 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:44,681 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:44,683 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,683 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,683 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,684 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,684 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,684 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,684 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,684 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,684 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,684 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41445,1676076840467': 2023-02-11 00:55:44,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:55:44,687 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 00:55:44,687 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:44,687 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:44,687 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,687 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:44,688 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,688 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,689 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,691 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,692 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,693 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,693 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,698 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,698 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:44,698 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:44,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,698 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,699 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,783 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 00:55:44,799 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:44,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:44,810 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:44,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:44,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,811 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:44,811 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,811 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 00:55:44,811 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 00:55:44,811 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,811 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,812 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,812 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,812 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,812 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,813 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,813 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:44,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,813 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,814 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,815 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,815 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,815 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:44,815 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:44,815 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,815 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:44,816 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,816 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. started... 2023-02-11 00:55:44,816 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,816 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,816 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,816 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,816 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,816 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 154356c6ea2dcb7760e343c32daf69e8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:44,817 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,819 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,819 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,819 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,819 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,819 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,824 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/cb98ead6ef774c048bfa000a87cb3bcd 2023-02-11 00:55:44,834 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/83cc53c5264e4cd099cad8a41ea7ae11 2023-02-11 00:55:44,837 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/cb98ead6ef774c048bfa000a87cb3bcd as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/cb98ead6ef774c048bfa000a87cb3bcd 2023-02-11 00:55:44,840 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/cb98ead6ef774c048bfa000a87cb3bcd, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:55:44,841 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/83cc53c5264e4cd099cad8a41ea7ae11 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/83cc53c5264e4cd099cad8a41ea7ae11 2023-02-11 00:55:44,844 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/83cc53c5264e4cd099cad8a41ea7ae11, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:55:44,844 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 154356c6ea2dcb7760e343c32daf69e8 in 28ms, sequenceid=58, compaction requested=true 2023-02-11 00:55:44,844 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 154356c6ea2dcb7760e343c32daf69e8: 2023-02-11 00:55:44,844 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:44,845 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:44,845 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:44,845 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,845 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,845 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,846 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,846 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,846 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,846 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 00:55:44,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10b993b8[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:44,848 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 00:55:44,848 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,849 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:44,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,849 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,849 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,849 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,849 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:44,849 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:44,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,850 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:44,850 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:44,850 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:44,851 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,851 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,852 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,852 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,852 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:44,853 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,852 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,853 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41445,1676076840467': 2023-02-11 00:55:44,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:55:44,855 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 00:55:44,855 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:44,855 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:44,855 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,855 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:44,856 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,856 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,856 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:44,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,856 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,856 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,857 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,857 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,857 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,857 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,858 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:44,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,859 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:44,860 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,860 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,862 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:44,862 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,862 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,863 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:44,863 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:44,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:44,862 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:44,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:44,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:44,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:44,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,963 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:44,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:44,973 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:44,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:44,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,974 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:44,974 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,974 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 00:55:44,974 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 00:55:44,975 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,975 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,976 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,976 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,976 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,976 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:44,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:44,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:44,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,976 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:44,977 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,977 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,977 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,978 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:44,978 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:44,978 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. started... 2023-02-11 00:55:44,978 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 154356c6ea2dcb7760e343c32daf69e8 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:44,980 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,981 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,981 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,982 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,982 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:44,982 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:44,986 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/68ea82e783c54e31bc9d3768efeaf085 2023-02-11 00:55:45,000 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e 2023-02-11 00:55:45,004 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/68ea82e783c54e31bc9d3768efeaf085 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/68ea82e783c54e31bc9d3768efeaf085 2023-02-11 00:55:45,008 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/68ea82e783c54e31bc9d3768efeaf085, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:55:45,008 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e 2023-02-11 00:55:45,012 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:55:45,013 INFO [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 154356c6ea2dcb7760e343c32daf69e8 in 35ms, sequenceid=71, compaction requested=true 2023-02-11 00:55:45,013 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 154356c6ea2dcb7760e343c32daf69e8: 2023-02-11 00:55:45,014 DEBUG [rs(jenkins-hbase4.apache.org,41445,1676076840467)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:45,014 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:45,014 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:45,014 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,014 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 00:55:45,014 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 00:55:45,015 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:45,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:45,015 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:45,016 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,016 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:45,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:45,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:45,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 00:55:45,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72dfd517[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:45,018 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 00:55:45,018 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,023 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,023 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:45,023 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:45,023 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:45,023 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:45,023 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:45,024 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:45,024 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:45,024 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,025 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 00:55:45,025 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 00:55:45,025 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:45,026 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:45,026 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:45,026 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:45,026 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:45,026 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:45,026 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:45,027 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 00:55:45,027 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,027 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:45,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,027 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 00:55:45,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:45,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:45,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41445,1676076840467': 2023-02-11 00:55:45,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 00:55:45,031 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 00:55:45,031 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:45,031 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:45,031 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,031 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:45,032 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,032 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:45,032 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:45,032 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,032 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:45,033 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:45,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:45,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:45,033 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,034 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,035 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:45,035 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,035 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:45,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 00:55:45,039 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:45,039 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:45,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:45,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:45,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:45,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:45,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,039 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:45,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:45,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:45,040 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:45,040 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:45,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:45,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:45,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:45,040 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:45,040 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,040 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,040 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:45,040 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,040 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 00:55:45,140 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:45,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:45,148 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,148 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=154356c6ea2dcb7760e343c32daf69e8, daughterA=35c9f23a7639c73d99e6e45886896274, daughterB=965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:45,149 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=154356c6ea2dcb7760e343c32daf69e8, daughterA=35c9f23a7639c73d99e6e45886896274, daughterB=965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:45,149 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=154356c6ea2dcb7760e343c32daf69e8, daughterA=35c9f23a7639c73d99e6e45886896274, daughterB=965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:45,149 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=154356c6ea2dcb7760e343c32daf69e8, daughterA=35c9f23a7639c73d99e6e45886896274, daughterB=965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:45,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=154356c6ea2dcb7760e343c32daf69e8, UNASSIGN}] 2023-02-11 00:55:45,151 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=154356c6ea2dcb7760e343c32daf69e8, UNASSIGN 2023-02-11 00:55:45,152 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=154356c6ea2dcb7760e343c32daf69e8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:45,152 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076945152"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076945152"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076945152"}]},"ts":"1676076945152"} 2023-02-11 00:55:45,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 154356c6ea2dcb7760e343c32daf69e8, server=jenkins-hbase4.apache.org,41445,1676076840467}] 2023-02-11 00:55:45,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 154356c6ea2dcb7760e343c32daf69e8, disabling compactions & flushes 2023-02-11 00:55:45,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:45,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:45,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. after waiting 0 ms 2023-02-11 00:55:45,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:45,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 154356c6ea2dcb7760e343c32daf69e8 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 00:55:45,314 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/99cdd730b6884c33bca4e9882dca6dd6 2023-02-11 00:55:45,725 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/344f79b7a08f4be287cfe9f85472b3a4 2023-02-11 00:55:45,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf1/99cdd730b6884c33bca4e9882dca6dd6 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/99cdd730b6884c33bca4e9882dca6dd6 2023-02-11 00:55:45,732 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/99cdd730b6884c33bca4e9882dca6dd6, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:55:45,732 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.tmp/cf2/344f79b7a08f4be287cfe9f85472b3a4 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/344f79b7a08f4be287cfe9f85472b3a4 2023-02-11 00:55:45,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/344f79b7a08f4be287cfe9f85472b3a4, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:55:45,736 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 154356c6ea2dcb7760e343c32daf69e8 in 432ms, sequenceid=82, compaction requested=true 2023-02-11 00:55:45,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 00:55:45,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:45,741 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. 2023-02-11 00:55:45,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 154356c6ea2dcb7760e343c32daf69e8: Waiting for close lock at 1676076945304Running coprocessor pre-close hooks at 1676076945304Disabling compacts and flushes for region at 1676076945304Disabling writes for close at 1676076945304Obtaining lock to block concurrent updates at 1676076945304Preparing flush snapshotting stores in 154356c6ea2dcb7760e343c32daf69e8 at 1676076945304Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676076945305 (+1 ms)Flushing stores of testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. at 1676076945305Flushing 154356c6ea2dcb7760e343c32daf69e8/cf1: creating writer at 1676076945307 (+2 ms)Flushing 154356c6ea2dcb7760e343c32daf69e8/cf1: appending metadata at 1676076945309 (+2 ms)Flushing 154356c6ea2dcb7760e343c32daf69e8/cf1: closing flushed file at 1676076945309Flushing 154356c6ea2dcb7760e343c32daf69e8/cf2: creating writer at 1676076945318 (+9 ms)Flushing 154356c6ea2dcb7760e343c32daf69e8/cf2: appending metadata at 1676076945319 (+1 ms)Flushing 154356c6ea2dcb7760e343c32daf69e8/cf2: closing flushed file at 1676076945319Flushing 154356c6ea2dcb7760e343c32daf69e8/cf1: reopening flushed file at 1676076945729 (+410 ms)Flushing 154356c6ea2dcb7760e343c32daf69e8/cf2: reopening flushed file at 1676076945733 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 154356c6ea2dcb7760e343c32daf69e8 in 432ms, sequenceid=82, compaction requested=true at 1676076945736 (+3 ms)Writing region close event to WAL at 1676076945737 (+1 ms)Running coprocessor post-close hooks at 1676076945740 (+3 ms)Closed at 1676076945741 (+1 ms) 2023-02-11 00:55:45,742 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,742 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=154356c6ea2dcb7760e343c32daf69e8, regionState=CLOSED 2023-02-11 00:55:45,742 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076945742"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076945742"}]},"ts":"1676076945742"} 2023-02-11 00:55:45,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-11 00:55:45,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 154356c6ea2dcb7760e343c32daf69e8, server=jenkins-hbase4.apache.org,41445,1676076840467 in 590 msec 2023-02-11 00:55:45,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-11 00:55:45,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=154356c6ea2dcb7760e343c32daf69e8, UNASSIGN in 593 msec 2023-02-11 00:55:45,754 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=154356c6ea2dcb7760e343c32daf69e8, threads=14 2023-02-11 00:55:45,754 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/0785c8f104de442d89d8df43e94e8452 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,754 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/2c29c4786e964973a476f7f4815fffb4 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,754 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/344f79b7a08f4be287cfe9f85472b3a4 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,754 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/38a019d9dcc447f4b48744ed8a7696e0 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,755 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/83cc53c5264e4cd099cad8a41ea7ae11 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,755 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,755 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/d9b50a36871d4122b0d32eafe4925241 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,755 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/5080b0776d74409e8d3f182c522f1fac for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,755 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/68ea82e783c54e31bc9d3768efeaf085 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,755 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/99cdd730b6884c33bca4e9882dca6dd6 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,755 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/ab4cb2200c634d93b6982048f726b48f for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,755 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/acaec320e3e34a90a5a7666c37739fc7 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,756 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/cb98ead6ef774c048bfa000a87cb3bcd for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,756 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/de066b44c1764306bee82d61a40154df for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,760 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e, top=false 2023-02-11 00:55:45,760 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/cb98ead6ef774c048bfa000a87cb3bcd, top=false 2023-02-11 00:55:45,760 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/ab4cb2200c634d93b6982048f726b48f, top=false 2023-02-11 00:55:45,770 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/0785c8f104de442d89d8df43e94e8452, top=false 2023-02-11 00:55:45,770 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/2c29c4786e964973a476f7f4815fffb4, top=false 2023-02-11 00:55:45,770 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/38a019d9dcc447f4b48744ed8a7696e0, top=false 2023-02-11 00:55:45,771 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/5080b0776d74409e8d3f182c522f1fac, top=false 2023-02-11 00:55:45,771 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/99cdd730b6884c33bca4e9882dca6dd6, top=false 2023-02-11 00:55:45,771 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/acaec320e3e34a90a5a7666c37739fc7, top=false 2023-02-11 00:55:45,771 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/83cc53c5264e4cd099cad8a41ea7ae11, top=false 2023-02-11 00:55:45,772 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/344f79b7a08f4be287cfe9f85472b3a4, top=false 2023-02-11 00:55:45,772 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/68ea82e783c54e31bc9d3768efeaf085, top=false 2023-02-11 00:55:45,776 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/d9b50a36871d4122b0d32eafe4925241, top=false 2023-02-11 00:55:45,778 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/de066b44c1764306bee82d61a40154df, top=false 2023-02-11 00:55:45,792 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cce3d8ca04b94d53b2e20f5b2f5ce99e for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,796 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-ab4cb2200c634d93b6982048f726b48f for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,798 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,798 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/ab4cb2200c634d93b6982048f726b48f for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,800 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-5080b0776d74409e8d3f182c522f1fac for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,800 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-68ea82e783c54e31bc9d3768efeaf085 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,801 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cb98ead6ef774c048bfa000a87cb3bcd for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,801 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-38a019d9dcc447f4b48744ed8a7696e0 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,801 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-de066b44c1764306bee82d61a40154df for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,804 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/5080b0776d74409e8d3f182c522f1fac for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,805 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/cb98ead6ef774c048bfa000a87cb3bcd for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,805 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/de066b44c1764306bee82d61a40154df for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,806 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/68ea82e783c54e31bc9d3768efeaf085 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,810 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/38a019d9dcc447f4b48744ed8a7696e0 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,818 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-344f79b7a08f4be287cfe9f85472b3a4 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,819 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-0785c8f104de442d89d8df43e94e8452 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,819 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-acaec320e3e34a90a5a7666c37739fc7 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,819 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-83cc53c5264e4cd099cad8a41ea7ae11 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,819 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-99cdd730b6884c33bca4e9882dca6dd6 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,819 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-d9b50a36871d4122b0d32eafe4925241 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,820 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/.splits/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-2c29c4786e964973a476f7f4815fffb4 for child: 35c9f23a7639c73d99e6e45886896274, parent: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,822 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/acaec320e3e34a90a5a7666c37739fc7 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,823 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/d9b50a36871d4122b0d32eafe4925241 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,823 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/83cc53c5264e4cd099cad8a41ea7ae11 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,824 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/99cdd730b6884c33bca4e9882dca6dd6 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,824 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/0785c8f104de442d89d8df43e94e8452 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,824 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/344f79b7a08f4be287cfe9f85472b3a4 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,825 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/2c29c4786e964973a476f7f4815fffb4 for region: 154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:45,825 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 154356c6ea2dcb7760e343c32daf69e8 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-11 00:55:46,282 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:55:46,284 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:55:46,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676076946285"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676076946285"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676076946285"}]},"ts":"1676076946285"} 2023-02-11 00:55:46,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076946285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076946285"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076946285"}]},"ts":"1676076946285"} 2023-02-11 00:55:46,286 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076946285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076946285"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076946285"}]},"ts":"1676076946285"} 2023-02-11 00:55:46,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=35c9f23a7639c73d99e6e45886896274, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=965259995dbc4950ffa8933d4874e66a, ASSIGN}] 2023-02-11 00:55:46,288 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=965259995dbc4950ffa8933d4874e66a, ASSIGN 2023-02-11 00:55:46,289 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=965259995dbc4950ffa8933d4874e66a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41445,1676076840467; forceNewPlan=false, retain=false 2023-02-11 00:55:46,289 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=35c9f23a7639c73d99e6e45886896274, ASSIGN 2023-02-11 00:55:46,290 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=35c9f23a7639c73d99e6e45886896274, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41445,1676076840467; forceNewPlan=false, retain=false 2023-02-11 00:55:46,439 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 00:55:46,441 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=965259995dbc4950ffa8933d4874e66a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:46,441 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076946441"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076946441"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076946441"}]},"ts":"1676076946441"} 2023-02-11 00:55:46,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 965259995dbc4950ffa8933d4874e66a, server=jenkins-hbase4.apache.org,41445,1676076840467}] 2023-02-11 00:55:46,444 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=35c9f23a7639c73d99e6e45886896274, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:46,444 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076946444"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076946444"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076946444"}]},"ts":"1676076946444"} 2023-02-11 00:55:46,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 35c9f23a7639c73d99e6e45886896274, server=jenkins-hbase4.apache.org,41445,1676076840467}] 2023-02-11 00:55:46,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a. 2023-02-11 00:55:46,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 965259995dbc4950ffa8933d4874e66a, NAME => 'testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.', STARTKEY => '71', ENDKEY => ''} 2023-02-11 00:55:46,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:46,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:46,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:46,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:46,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:46,597 INFO [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:46,598 DEBUG [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/cf1 2023-02-11 00:55:46,598 DEBUG [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/cf1 2023-02-11 00:55:46,599 INFO [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 965259995dbc4950ffa8933d4874e66a columnFamilyName cf1 2023-02-11 00:55:46,599 INFO [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] regionserver.HStore(310): Store=965259995dbc4950ffa8933d4874e66a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:46,599 INFO [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:46,600 DEBUG [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/cf2 2023-02-11 00:55:46,600 DEBUG [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/cf2 2023-02-11 00:55:46,600 INFO [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 965259995dbc4950ffa8933d4874e66a columnFamilyName cf2 2023-02-11 00:55:46,601 INFO [StoreOpener-965259995dbc4950ffa8933d4874e66a-1] regionserver.HStore(310): Store=965259995dbc4950ffa8933d4874e66a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:46,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:46,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:46,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:46,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:46,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 965259995dbc4950ffa8933d4874e66a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63135147, jitterRate=-0.05921299755573273}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:46,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 965259995dbc4950ffa8933d4874e66a: Running coprocessor pre-open hook at 1676076946596Writing region info on filesystem at 1676076946596Initializing all the Stores at 1676076946597 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076946597Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076946597Cleaning up temporary data from old regions at 1676076946602 (+5 ms)Cleaning up detritus from prior splits at 1676076946602Running coprocessor post-open hooks at 1676076946604 (+2 ms)Region opened successfully at 1676076946604 2023-02-11 00:55:46,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a., pid=133, masterSystemTime=1676076946593 2023-02-11 00:55:46,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a. 2023-02-11 00:55:46,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a. 2023-02-11 00:55:46,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:46,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 35c9f23a7639c73d99e6e45886896274, NAME => 'testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.', STARTKEY => '', ENDKEY => '71'} 2023-02-11 00:55:46,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=965259995dbc4950ffa8933d4874e66a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:46,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:46,606 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076946606"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076946606"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076946606"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076946606"}]},"ts":"1676076946606"} 2023-02-11 00:55:46,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:46,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:46,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:46,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:46,607 INFO [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:46,608 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1 2023-02-11 00:55:46,608 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1 2023-02-11 00:55:46,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-11 00:55:46,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 965259995dbc4950ffa8933d4874e66a, server=jenkins-hbase4.apache.org,41445,1676076840467 in 165 msec 2023-02-11 00:55:46,608 INFO [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35c9f23a7639c73d99e6e45886896274 columnFamilyName cf1 2023-02-11 00:55:46,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=965259995dbc4950ffa8933d4874e66a, ASSIGN in 321 msec 2023-02-11 00:55:46,613 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-5080b0776d74409e8d3f182c522f1fac 2023-02-11 00:55:46,616 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-68ea82e783c54e31bc9d3768efeaf085 2023-02-11 00:55:46,618 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-99cdd730b6884c33bca4e9882dca6dd6 2023-02-11 00:55:46,620 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-ab4cb2200c634d93b6982048f726b48f 2023-02-11 00:55:46,622 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-acaec320e3e34a90a5a7666c37739fc7 2023-02-11 00:55:46,625 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cb98ead6ef774c048bfa000a87cb3bcd 2023-02-11 00:55:46,626 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-de066b44c1764306bee82d61a40154df 2023-02-11 00:55:46,627 INFO [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(310): Store=35c9f23a7639c73d99e6e45886896274/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:46,627 INFO [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:46,627 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2 2023-02-11 00:55:46,627 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2 2023-02-11 00:55:46,628 INFO [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35c9f23a7639c73d99e6e45886896274 columnFamilyName cf2 2023-02-11 00:55:46,631 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-0785c8f104de442d89d8df43e94e8452 2023-02-11 00:55:46,634 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-2c29c4786e964973a476f7f4815fffb4 2023-02-11 00:55:46,636 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-344f79b7a08f4be287cfe9f85472b3a4 2023-02-11 00:55:46,638 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-38a019d9dcc447f4b48744ed8a7696e0 2023-02-11 00:55:46,640 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-83cc53c5264e4cd099cad8a41ea7ae11 2023-02-11 00:55:46,642 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cce3d8ca04b94d53b2e20f5b2f5ce99e 2023-02-11 00:55:46,644 DEBUG [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-d9b50a36871d4122b0d32eafe4925241 2023-02-11 00:55:46,644 INFO [StoreOpener-35c9f23a7639c73d99e6e45886896274-1] regionserver.HStore(310): Store=35c9f23a7639c73d99e6e45886896274/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:46,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:46,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:46,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:46,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:46,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 35c9f23a7639c73d99e6e45886896274; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65840270, jitterRate=-0.018903523683547974}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:46,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 35c9f23a7639c73d99e6e45886896274: Running coprocessor pre-open hook at 1676076946606Writing region info on filesystem at 1676076946606Initializing all the Stores at 1676076946607 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076946607Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076946607Cleaning up temporary data from old regions at 1676076946646 (+39 ms)Cleaning up detritus from prior splits at 1676076946647 (+1 ms)Running coprocessor post-open hooks at 1676076946648 (+1 ms)Region opened successfully at 1676076946648 2023-02-11 00:55:46,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274., pid=134, masterSystemTime=1676076946593 2023-02-11 00:55:46,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:46,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 00:55:46,650 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:46,650 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:46,650 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 00:55:46,650 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 00:55:46,650 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] regionserver.HStore(1912): 35c9f23a7639c73d99e6e45886896274/cf1 is initiating minor compaction (all files) 2023-02-11 00:55:46,651 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] regionserver.HStore(1912): 35c9f23a7639c73d99e6e45886896274/cf2 is initiating minor compaction (all files) 2023-02-11 00:55:46,651 INFO [RS:2;jenkins-hbase4:41445-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 35c9f23a7639c73d99e6e45886896274/cf1 in testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:46,651 INFO [RS:2;jenkins-hbase4:41445-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 35c9f23a7639c73d99e6e45886896274/cf2 in testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:46,651 INFO [RS:2;jenkins-hbase4:41445-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-5080b0776d74409e8d3f182c522f1fac, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-ab4cb2200c634d93b6982048f726b48f, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-de066b44c1764306bee82d61a40154df, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-acaec320e3e34a90a5a7666c37739fc7, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cb98ead6ef774c048bfa000a87cb3bcd, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-68ea82e783c54e31bc9d3768efeaf085, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-99cdd730b6884c33bca4e9882dca6dd6] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/.tmp, totalSize=34.9 K 2023-02-11 00:55:46,651 INFO [RS:2;jenkins-hbase4:41445-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-2c29c4786e964973a476f7f4815fffb4, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-d9b50a36871d4122b0d32eafe4925241, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-38a019d9dcc447f4b48744ed8a7696e0, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-0785c8f104de442d89d8df43e94e8452, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-83cc53c5264e4cd099cad8a41ea7ae11, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cce3d8ca04b94d53b2e20f5b2f5ce99e, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-344f79b7a08f4be287cfe9f85472b3a4] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/.tmp, totalSize=34.9 K 2023-02-11 00:55:46,651 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-5080b0776d74409e8d3f182c522f1fac, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076943311 2023-02-11 00:55:46,651 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-2c29c4786e964973a476f7f4815fffb4, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076943311 2023-02-11 00:55:46,651 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-ab4cb2200c634d93b6982048f726b48f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076943892 2023-02-11 00:55:46,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:46,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:46,652 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-d9b50a36871d4122b0d32eafe4925241, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076943892 2023-02-11 00:55:46,652 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=35c9f23a7639c73d99e6e45886896274, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:46,652 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-de066b44c1764306bee82d61a40154df, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076944055 2023-02-11 00:55:46,652 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076946652"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076946652"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076946652"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076946652"}]},"ts":"1676076946652"} 2023-02-11 00:55:46,652 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-38a019d9dcc447f4b48744ed8a7696e0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076944055 2023-02-11 00:55:46,652 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-acaec320e3e34a90a5a7666c37739fc7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076944622 2023-02-11 00:55:46,653 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-0785c8f104de442d89d8df43e94e8452, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076944622 2023-02-11 00:55:46,653 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cb98ead6ef774c048bfa000a87cb3bcd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076944800 2023-02-11 00:55:46,653 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-83cc53c5264e4cd099cad8a41ea7ae11, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076944800 2023-02-11 00:55:46,653 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-68ea82e783c54e31bc9d3768efeaf085, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076944964 2023-02-11 00:55:46,653 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cce3d8ca04b94d53b2e20f5b2f5ce99e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076944964 2023-02-11 00:55:46,654 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-99cdd730b6884c33bca4e9882dca6dd6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076945141 2023-02-11 00:55:46,654 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-344f79b7a08f4be287cfe9f85472b3a4, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076945141 2023-02-11 00:55:46,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-11 00:55:46,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 35c9f23a7639c73d99e6e45886896274, server=jenkins-hbase4.apache.org,41445,1676076840467 in 208 msec 2023-02-11 00:55:46,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-11 00:55:46,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=35c9f23a7639c73d99e6e45886896274, ASSIGN in 368 msec 2023-02-11 00:55:46,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=154356c6ea2dcb7760e343c32daf69e8, daughterA=35c9f23a7639c73d99e6e45886896274, daughterB=965259995dbc4950ffa8933d4874e66a in 1.5090 sec 2023-02-11 00:55:46,674 INFO [RS:2;jenkins-hbase4:41445-shortCompactions-0] throttle.PressureAwareThroughputController(145): 35c9f23a7639c73d99e6e45886896274#cf1#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:46,674 INFO [RS:2;jenkins-hbase4:41445-longCompactions-0] throttle.PressureAwareThroughputController(145): 35c9f23a7639c73d99e6e45886896274#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:46,684 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/.tmp/cf1/0fa66604be254e5d88578b4a05c33d29 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/0fa66604be254e5d88578b4a05c33d29 2023-02-11 00:55:46,684 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/.tmp/cf2/41a11be9bc304d19af3b34717f6b24cf as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/41a11be9bc304d19af3b34717f6b24cf 2023-02-11 00:55:46,688 INFO [RS:2;jenkins-hbase4:41445-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 35c9f23a7639c73d99e6e45886896274/cf1 of 35c9f23a7639c73d99e6e45886896274 into 0fa66604be254e5d88578b4a05c33d29(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:46,688 INFO [RS:2;jenkins-hbase4:41445-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 35c9f23a7639c73d99e6e45886896274/cf2 of 35c9f23a7639c73d99e6e45886896274 into 41a11be9bc304d19af3b34717f6b24cf(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:46,688 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 35c9f23a7639c73d99e6e45886896274: 2023-02-11 00:55:46,688 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 35c9f23a7639c73d99e6e45886896274: 2023-02-11 00:55:46,688 INFO [RS:2;jenkins-hbase4:41445-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274., storeName=35c9f23a7639c73d99e6e45886896274/cf1, priority=9, startTime=1676076946649; duration=0sec 2023-02-11 00:55:46,688 INFO [RS:2;jenkins-hbase4:41445-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274., storeName=35c9f23a7639c73d99e6e45886896274/cf2, priority=9, startTime=1676076946649; duration=0sec 2023-02-11 00:55:46,689 DEBUG [RS:2;jenkins-hbase4:41445-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:46,689 DEBUG [RS:2;jenkins-hbase4:41445-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:46,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41445] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:44478 deadline: 1676077006900, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8. is not online on jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:47,010 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-11 00:55:47,013 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:47,014 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,014 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 00:55:47,018 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076947017"}]},"ts":"1676076947017"} 2023-02-11 00:55:47,018 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-11 00:55:47,020 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-11 00:55:47,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=35c9f23a7639c73d99e6e45886896274, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=965259995dbc4950ffa8933d4874e66a, UNASSIGN}] 2023-02-11 00:55:47,021 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=965259995dbc4950ffa8933d4874e66a, UNASSIGN 2023-02-11 00:55:47,022 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=965259995dbc4950ffa8933d4874e66a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:47,022 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076947022"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076947022"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076947022"}]},"ts":"1676076947022"} 2023-02-11 00:55:47,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 965259995dbc4950ffa8933d4874e66a, server=jenkins-hbase4.apache.org,41445,1676076840467}] 2023-02-11 00:55:47,025 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=35c9f23a7639c73d99e6e45886896274, UNASSIGN 2023-02-11 00:55:47,025 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=35c9f23a7639c73d99e6e45886896274, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:47,025 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076947025"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076947025"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076947025"}]},"ts":"1676076947025"} 2023-02-11 00:55:47,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 35c9f23a7639c73d99e6e45886896274, server=jenkins-hbase4.apache.org,41445,1676076840467}] 2023-02-11 00:55:47,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 00:55:47,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:47,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 35c9f23a7639c73d99e6e45886896274, disabling compactions & flushes 2023-02-11 00:55:47,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:47,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:47,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. after waiting 0 ms 2023-02-11 00:55:47,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:47,175 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-5080b0776d74409e8d3f182c522f1fac, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-ab4cb2200c634d93b6982048f726b48f, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-de066b44c1764306bee82d61a40154df, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-acaec320e3e34a90a5a7666c37739fc7, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cb98ead6ef774c048bfa000a87cb3bcd, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-68ea82e783c54e31bc9d3768efeaf085, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-99cdd730b6884c33bca4e9882dca6dd6] to archive 2023-02-11 00:55:47,176 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:47,178 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-5080b0776d74409e8d3f182c522f1fac to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-5080b0776d74409e8d3f182c522f1fac 2023-02-11 00:55:47,178 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-ab4cb2200c634d93b6982048f726b48f to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-ab4cb2200c634d93b6982048f726b48f 2023-02-11 00:55:47,179 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-de066b44c1764306bee82d61a40154df to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-de066b44c1764306bee82d61a40154df 2023-02-11 00:55:47,180 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-acaec320e3e34a90a5a7666c37739fc7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-acaec320e3e34a90a5a7666c37739fc7 2023-02-11 00:55:47,181 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cb98ead6ef774c048bfa000a87cb3bcd to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cb98ead6ef774c048bfa000a87cb3bcd 2023-02-11 00:55:47,182 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-68ea82e783c54e31bc9d3768efeaf085 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-68ea82e783c54e31bc9d3768efeaf085 2023-02-11 00:55:47,183 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-99cdd730b6884c33bca4e9882dca6dd6 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-99cdd730b6884c33bca4e9882dca6dd6 2023-02-11 00:55:47,184 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-2c29c4786e964973a476f7f4815fffb4, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-d9b50a36871d4122b0d32eafe4925241, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-38a019d9dcc447f4b48744ed8a7696e0, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-0785c8f104de442d89d8df43e94e8452, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-83cc53c5264e4cd099cad8a41ea7ae11, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cce3d8ca04b94d53b2e20f5b2f5ce99e, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-344f79b7a08f4be287cfe9f85472b3a4] to archive 2023-02-11 00:55:47,185 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:47,187 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-2c29c4786e964973a476f7f4815fffb4 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-2c29c4786e964973a476f7f4815fffb4 2023-02-11 00:55:47,187 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-d9b50a36871d4122b0d32eafe4925241 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-d9b50a36871d4122b0d32eafe4925241 2023-02-11 00:55:47,188 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-38a019d9dcc447f4b48744ed8a7696e0 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-38a019d9dcc447f4b48744ed8a7696e0 2023-02-11 00:55:47,190 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-0785c8f104de442d89d8df43e94e8452 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-0785c8f104de442d89d8df43e94e8452 2023-02-11 00:55:47,190 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-83cc53c5264e4cd099cad8a41ea7ae11 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-83cc53c5264e4cd099cad8a41ea7ae11 2023-02-11 00:55:47,191 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cce3d8ca04b94d53b2e20f5b2f5ce99e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-cce3d8ca04b94d53b2e20f5b2f5ce99e 2023-02-11 00:55:47,192 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-344f79b7a08f4be287cfe9f85472b3a4 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/testSplitTableRegionEmptyDaughter=154356c6ea2dcb7760e343c32daf69e8-344f79b7a08f4be287cfe9f85472b3a4 2023-02-11 00:55:47,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:55:47,195 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:47,196 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274. 2023-02-11 00:55:47,196 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 35c9f23a7639c73d99e6e45886896274: Waiting for close lock at 1676076947174Running coprocessor pre-close hooks at 1676076947174Disabling compacts and flushes for region at 1676076947174Disabling writes for close at 1676076947175 (+1 ms)Writing region close event to WAL at 1676076947193 (+18 ms)Running coprocessor post-close hooks at 1676076947195 (+2 ms)Closed at 1676076947196 (+1 ms) 2023-02-11 00:55:47,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:47,197 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:47,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 965259995dbc4950ffa8933d4874e66a, disabling compactions & flushes 2023-02-11 00:55:47,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a. 2023-02-11 00:55:47,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a. 2023-02-11 00:55:47,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a. after waiting 0 ms 2023-02-11 00:55:47,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a. 2023-02-11 00:55:47,198 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=35c9f23a7639c73d99e6e45886896274, regionState=CLOSED 2023-02-11 00:55:47,198 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076947198"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076947198"}]},"ts":"1676076947198"} 2023-02-11 00:55:47,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-11 00:55:47,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-11 00:55:47,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 35c9f23a7639c73d99e6e45886896274, server=jenkins-hbase4.apache.org,41445,1676076840467 in 173 msec 2023-02-11 00:55:47,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:47,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a. 2023-02-11 00:55:47,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 965259995dbc4950ffa8933d4874e66a: Waiting for close lock at 1676076947197Running coprocessor pre-close hooks at 1676076947197Disabling compacts and flushes for region at 1676076947197Disabling writes for close at 1676076947198 (+1 ms)Writing region close event to WAL at 1676076947198Running coprocessor post-close hooks at 1676076947201 (+3 ms)Closed at 1676076947201 2023-02-11 00:55:47,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=35c9f23a7639c73d99e6e45886896274, UNASSIGN in 179 msec 2023-02-11 00:55:47,202 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:47,202 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=965259995dbc4950ffa8933d4874e66a, regionState=CLOSED 2023-02-11 00:55:47,202 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676076947202"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076947202"}]},"ts":"1676076947202"} 2023-02-11 00:55:47,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-11 00:55:47,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 965259995dbc4950ffa8933d4874e66a, server=jenkins-hbase4.apache.org,41445,1676076840467 in 180 msec 2023-02-11 00:55:47,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-11 00:55:47,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=965259995dbc4950ffa8933d4874e66a, UNASSIGN in 183 msec 2023-02-11 00:55:47,206 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076947206"}]},"ts":"1676076947206"} 2023-02-11 00:55:47,207 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-11 00:55:47,210 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-11 00:55:47,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 195 msec 2023-02-11 00:55:47,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 00:55:47,319 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-11 00:55:47,320 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,322 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,323 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-11 00:55:47,327 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:47,327 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:47,327 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:47,329 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/recovered.edits] 2023-02-11 00:55:47,329 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/recovered.edits] 2023-02-11 00:55:47,329 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/recovered.edits] 2023-02-11 00:55:47,332 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/5080b0776d74409e8d3f182c522f1fac to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/5080b0776d74409e8d3f182c522f1fac 2023-02-11 00:55:47,332 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/0fa66604be254e5d88578b4a05c33d29 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf1/0fa66604be254e5d88578b4a05c33d29 2023-02-11 00:55:47,333 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/68ea82e783c54e31bc9d3768efeaf085 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/68ea82e783c54e31bc9d3768efeaf085 2023-02-11 00:55:47,334 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/recovered.edits/88.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a/recovered.edits/88.seqid 2023-02-11 00:55:47,334 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/99cdd730b6884c33bca4e9882dca6dd6 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/99cdd730b6884c33bca4e9882dca6dd6 2023-02-11 00:55:47,334 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/41a11be9bc304d19af3b34717f6b24cf to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/cf2/41a11be9bc304d19af3b34717f6b24cf 2023-02-11 00:55:47,334 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/965259995dbc4950ffa8933d4874e66a 2023-02-11 00:55:47,335 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/ab4cb2200c634d93b6982048f726b48f to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/ab4cb2200c634d93b6982048f726b48f 2023-02-11 00:55:47,336 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/acaec320e3e34a90a5a7666c37739fc7 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/acaec320e3e34a90a5a7666c37739fc7 2023-02-11 00:55:47,336 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274/recovered.edits/90.seqid 2023-02-11 00:55:47,337 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/35c9f23a7639c73d99e6e45886896274 2023-02-11 00:55:47,337 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/cb98ead6ef774c048bfa000a87cb3bcd to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/cb98ead6ef774c048bfa000a87cb3bcd 2023-02-11 00:55:47,338 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/de066b44c1764306bee82d61a40154df to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf1/de066b44c1764306bee82d61a40154df 2023-02-11 00:55:47,340 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/0785c8f104de442d89d8df43e94e8452 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/0785c8f104de442d89d8df43e94e8452 2023-02-11 00:55:47,341 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/2c29c4786e964973a476f7f4815fffb4 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/2c29c4786e964973a476f7f4815fffb4 2023-02-11 00:55:47,342 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/344f79b7a08f4be287cfe9f85472b3a4 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/344f79b7a08f4be287cfe9f85472b3a4 2023-02-11 00:55:47,343 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/38a019d9dcc447f4b48744ed8a7696e0 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/38a019d9dcc447f4b48744ed8a7696e0 2023-02-11 00:55:47,343 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/83cc53c5264e4cd099cad8a41ea7ae11 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/83cc53c5264e4cd099cad8a41ea7ae11 2023-02-11 00:55:47,344 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/cce3d8ca04b94d53b2e20f5b2f5ce99e 2023-02-11 00:55:47,345 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/d9b50a36871d4122b0d32eafe4925241 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/cf2/d9b50a36871d4122b0d32eafe4925241 2023-02-11 00:55:47,347 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/recovered.edits/85.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8/recovered.edits/85.seqid 2023-02-11 00:55:47,347 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegionEmptyDaughter/154356c6ea2dcb7760e343c32daf69e8 2023-02-11 00:55:47,347 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-11 00:55:47,349 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,350 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-11 00:55:47,352 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-11 00:55:47,353 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,353 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-11 00:55:47,353 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076947353"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:47,353 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076947353"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:47,353 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076947353"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:47,355 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 00:55:47,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 154356c6ea2dcb7760e343c32daf69e8, NAME => 'testSplitTableRegionEmptyDaughter,,1676076942803.154356c6ea2dcb7760e343c32daf69e8.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 35c9f23a7639c73d99e6e45886896274, NAME => 'testSplitTableRegionEmptyDaughter,,1676076945148.35c9f23a7639c73d99e6e45886896274.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 965259995dbc4950ffa8933d4874e66a, NAME => 'testSplitTableRegionEmptyDaughter,71,1676076945148.965259995dbc4950ffa8933d4874e66a.', STARTKEY => '71', ENDKEY => ''}] 2023-02-11 00:55:47,355 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-11 00:55:47,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076947355"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:47,356 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-11 00:55:47,358 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 00:55:47,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 38 msec 2023-02-11 00:55:47,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-11 00:55:47,424 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-11 00:55:47,437 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 373) - Thread LEAK? -, OpenFileDescriptor=835 (was 831) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=103 (was 103), ProcessCount=168 (was 168), AvailableMemoryMB=2934 (was 2941) 2023-02-11 00:55:47,450 INFO [Listener at localhost/35701] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=103, ProcessCount=168, AvailableMemoryMB=2933 2023-02-11 00:55:47,451 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:47,454 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 00:55:47,454 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 00:55:47,455 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-11 00:55:47,457 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 00:55:47,457 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 00:55:47,458 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,459 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed empty. 2023-02-11 00:55:47,459 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-11 00:55:47,469 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-11 00:55:47,470 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => e4748df108f96490e961347f1b71a1ed, NAME => 'testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp 2023-02-11 00:55:47,477 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:47,477 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing e4748df108f96490e961347f1b71a1ed, disabling compactions & flushes 2023-02-11 00:55:47,477 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,477 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,477 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. after waiting 0 ms 2023-02-11 00:55:47,477 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,477 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,477 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for e4748df108f96490e961347f1b71a1ed: Waiting for close lock at 1676076947477Disabling compacts and flushes for region at 1676076947477Disabling writes for close at 1676076947477Writing region close event to WAL at 1676076947477Closed at 1676076947477 2023-02-11 00:55:47,479 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 00:55:47,479 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076947479"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076947479"}]},"ts":"1676076947479"} 2023-02-11 00:55:47,480 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 00:55:47,481 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 00:55:47,481 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076947481"}]},"ts":"1676076947481"} 2023-02-11 00:55:47,482 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-11 00:55:47,485 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 00:55:47,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 00:55:47,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 00:55:47,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 00:55:47,486 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 00:55:47,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e4748df108f96490e961347f1b71a1ed, ASSIGN}] 2023-02-11 00:55:47,487 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e4748df108f96490e961347f1b71a1ed, ASSIGN 2023-02-11 00:55:47,488 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=e4748df108f96490e961347f1b71a1ed, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33641,1676076840386; forceNewPlan=false, retain=false 2023-02-11 00:55:47,638 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 00:55:47,639 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e4748df108f96490e961347f1b71a1ed, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:47,639 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076947639"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076947639"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076947639"}]},"ts":"1676076947639"} 2023-02-11 00:55:47,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure e4748df108f96490e961347f1b71a1ed, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:55:47,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e4748df108f96490e961347f1b71a1ed, NAME => 'testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.', STARTKEY => '', ENDKEY => ''} 2023-02-11 00:55:47,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:47,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:47,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,799 INFO [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,800 DEBUG [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1 2023-02-11 00:55:47,800 DEBUG [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1 2023-02-11 00:55:47,801 INFO [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e4748df108f96490e961347f1b71a1ed columnFamilyName cf1 2023-02-11 00:55:47,801 INFO [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] regionserver.HStore(310): Store=e4748df108f96490e961347f1b71a1ed/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:47,801 INFO [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,802 DEBUG [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2 2023-02-11 00:55:47,802 DEBUG [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2 2023-02-11 00:55:47,803 INFO [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e4748df108f96490e961347f1b71a1ed columnFamilyName cf2 2023-02-11 00:55:47,803 INFO [StoreOpener-e4748df108f96490e961347f1b71a1ed-1] regionserver.HStore(310): Store=e4748df108f96490e961347f1b71a1ed/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:47,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:47,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:47,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 00:55:47,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e4748df108f96490e961347f1b71a1ed; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72377492, jitterRate=0.07850867509841919}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:47,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e4748df108f96490e961347f1b71a1ed: Running coprocessor pre-open hook at 1676076947798Writing region info on filesystem at 1676076947798Initializing all the Stores at 1676076947798Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076947799 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076947799Cleaning up temporary data from old regions at 1676076947805 (+6 ms)Cleaning up detritus from prior splits at 1676076947805Running coprocessor post-open hooks at 1676076947808 (+3 ms)Region opened successfully at 1676076947808 2023-02-11 00:55:47,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed., pid=143, masterSystemTime=1676076947791 2023-02-11 00:55:47,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,811 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e4748df108f96490e961347f1b71a1ed, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:47,811 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076947810"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076947810"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076947810"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076947810"}]},"ts":"1676076947810"} 2023-02-11 00:55:47,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-11 00:55:47,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure e4748df108f96490e961347f1b71a1ed, server=jenkins-hbase4.apache.org,33641,1676076840386 in 172 msec 2023-02-11 00:55:47,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-11 00:55:47,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e4748df108f96490e961347f1b71a1ed, ASSIGN in 327 msec 2023-02-11 00:55:47,816 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 00:55:47,816 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076947816"}]},"ts":"1676076947816"} 2023-02-11 00:55:47,817 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-11 00:55:47,819 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 00:55:47,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 364 msec 2023-02-11 00:55:47,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:47,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 00:55:47,963 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 00:55:47,963 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:47,963 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 00:55:47,963 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 00:55:47,964 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:47,964 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:47,965 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:47,965 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:47,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:47,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:47,965 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:47,965 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:47,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:47,965 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:47,965 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:47,965 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:47,966 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:47,966 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:47,966 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:47,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:47,967 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:47,967 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. started... 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:47,967 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:47,968 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4748df108f96490e961347f1b71a1ed 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 00:55:47,968 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:47,968 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:47,969 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:47,970 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:47,971 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:47,971 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:47,971 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:47,971 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:47,976 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/706969790a704660903b315c07221fbf 2023-02-11 00:55:47,988 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/4a78315cfa6d462786efac5e9bc967b3 2023-02-11 00:55:47,991 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/706969790a704660903b315c07221fbf as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf 2023-02-11 00:55:47,994 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:47,995 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/4a78315cfa6d462786efac5e9bc967b3 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3 2023-02-11 00:55:47,997 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:47,998 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e4748df108f96490e961347f1b71a1ed in 30ms, sequenceid=6, compaction requested=false 2023-02-11 00:55:47,998 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-11 00:55:47,998 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4748df108f96490e961347f1b71a1ed: 2023-02-11 00:55:47,998 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:47,999 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,000 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,000 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,000 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,000 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 00:55:48,002 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3da481f4[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:48,002 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 00:55:48,002 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,003 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,003 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,003 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,003 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,004 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,004 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:48,005 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,005 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,005 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,005 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,011 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,011 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,011 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,012 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,012 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,012 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:55:48,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 00:55:48,014 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 00:55:48,014 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:48,014 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:48,014 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 00:55:48,014 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:48,016 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,016 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,016 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,016 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,017 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,019 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,019 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,019 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,022 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,022 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 00:55:48,022 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:48,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,022 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:48,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,023 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,108 ERROR [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-74375045-172.31.14.131-1676076836114 (Datanode Uuid 0b950d92-fb4b-4a88-add0-dd4dcb24d701) service to localhost/127.0.0.1:46775 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 00:55:48,123 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:48,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:48,133 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:48,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:48,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 00:55:48,135 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 00:55:48,135 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,135 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 00:55:48,135 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 00:55:48,135 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,135 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,136 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,136 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,136 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,136 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,136 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,137 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:48,138 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:48,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,138 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,138 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. started... 2023-02-11 00:55:48,138 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4748df108f96490e961347f1b71a1ed 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:48,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,139 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,139 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,139 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,139 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,139 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,141 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,141 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,141 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,141 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,144 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,144 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,149 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/886d54b9c7d94704bb768b141a9fa076 2023-02-11 00:55:48,159 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/0c41a68b7bba4020abcb30eeb38b8b9d 2023-02-11 00:55:48,163 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/886d54b9c7d94704bb768b141a9fa076 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076 2023-02-11 00:55:48,166 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:55:48,166 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/0c41a68b7bba4020abcb30eeb38b8b9d as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d 2023-02-11 00:55:48,170 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 00:55:48,171 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4748df108f96490e961347f1b71a1ed in 33ms, sequenceid=19, compaction requested=false 2023-02-11 00:55:48,171 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4748df108f96490e961347f1b71a1ed: 2023-02-11 00:55:48,171 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,171 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:48,171 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:48,171 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,171 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,171 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,173 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 00:55:48,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b0b2d53[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:48,175 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 00:55:48,175 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,176 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:48,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,177 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,177 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,177 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:48,176 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,177 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:48,177 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,177 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,177 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:48,179 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,179 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,179 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,180 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,180 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,180 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,180 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,180 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,180 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,180 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:55:48,183 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 00:55:48,183 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 00:55:48,184 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:48,184 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:48,184 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 00:55:48,184 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:48,185 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,185 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,185 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,185 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,185 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,185 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,186 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,185 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,186 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,186 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,188 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,188 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,189 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,192 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,192 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 00:55:48,193 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,192 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,193 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:48,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,193 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,193 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:48,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,193 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,193 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,194 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,194 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,194 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,194 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,194 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,293 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:48,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:48,304 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:48,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:48,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 00:55:48,305 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 00:55:48,305 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,305 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 00:55:48,305 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 00:55:48,306 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,306 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,307 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,307 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,307 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,307 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,307 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,308 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,308 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,308 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,308 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,308 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,308 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:48,309 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:48,309 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,309 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. started... 2023-02-11 00:55:48,309 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4748df108f96490e961347f1b71a1ed 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:48,310 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,314 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,314 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,314 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,315 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,315 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,318 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/c6c000de86b24b5ebdafb3d4d4ef980f 2023-02-11 00:55:48,327 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/f37e6e300c4e4304b11c15d257761edb 2023-02-11 00:55:48,331 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/c6c000de86b24b5ebdafb3d4d4ef980f as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f 2023-02-11 00:55:48,333 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:55:48,334 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/f37e6e300c4e4304b11c15d257761edb as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb 2023-02-11 00:55:48,337 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 00:55:48,337 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4748df108f96490e961347f1b71a1ed in 28ms, sequenceid=32, compaction requested=true 2023-02-11 00:55:48,337 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4748df108f96490e961347f1b71a1ed: 2023-02-11 00:55:48,338 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,339 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,339 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,339 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,339 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 00:55:48,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f341d9[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:48,341 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 00:55:48,341 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,342 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,342 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:48,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,342 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:48,342 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,342 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,342 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:48,342 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,342 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,343 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:48,343 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,344 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,344 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,347 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,347 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,347 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,347 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,347 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,347 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,347 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:55:48,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 00:55:48,350 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 00:55:48,350 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:48,350 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:48,350 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 00:55:48,350 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:48,352 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,352 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,352 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,352 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,353 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,353 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,355 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,355 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,355 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,358 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 00:55:48,358 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:48,358 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,358 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:48,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,359 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,459 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:48,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:48,469 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:48,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:48,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 00:55:48,471 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 00:55:48,471 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,471 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 00:55:48,471 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 00:55:48,471 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,471 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,472 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,472 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,472 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,472 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,473 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,473 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,474 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,474 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,474 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,474 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,474 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,475 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:48,475 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:48,475 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,475 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. started... 2023-02-11 00:55:48,475 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4748df108f96490e961347f1b71a1ed 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:48,477 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,477 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,477 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,477 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,478 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,478 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,482 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/54ba7f21176849db98b8b14273a070d4 2023-02-11 00:55:48,494 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/4ae7ef72c25747cabcf910bbcb5835e5 2023-02-11 00:55:48,497 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/54ba7f21176849db98b8b14273a070d4 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4 2023-02-11 00:55:48,500 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:55:48,501 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/4ae7ef72c25747cabcf910bbcb5835e5 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5 2023-02-11 00:55:48,503 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 00:55:48,504 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4748df108f96490e961347f1b71a1ed in 29ms, sequenceid=45, compaction requested=true 2023-02-11 00:55:48,504 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4748df108f96490e961347f1b71a1ed: 2023-02-11 00:55:48,504 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,504 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,506 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,506 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,506 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,506 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 00:55:48,508 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b085b5d[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:48,508 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 00:55:48,508 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:48,509 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,510 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:48,509 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:48,512 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,512 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,512 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,512 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,512 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,512 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,513 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,513 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,513 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,513 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:55:48,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 00:55:48,516 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 00:55:48,516 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:48,516 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:48,516 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 00:55:48,516 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:48,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,517 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,525 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,526 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,526 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,528 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,529 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,529 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,536 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,536 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 00:55:48,536 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:48,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,536 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,537 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,537 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:48,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,537 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:48,637 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:48,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:48,649 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:48,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:48,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 00:55:48,650 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 00:55:48,650 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,650 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 00:55:48,650 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 00:55:48,651 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,651 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,652 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,652 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,652 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,652 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,653 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,653 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:48,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,654 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,654 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,654 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,654 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,654 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,654 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,654 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,654 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,654 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,654 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,654 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,654 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,655 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,655 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. started... 2023-02-11 00:55:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,655 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4748df108f96490e961347f1b71a1ed 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:48,656 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,657 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,657 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,657 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,657 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,657 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,665 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/65254bf47f1e46cb9650b4a5194f233f 2023-02-11 00:55:48,677 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/bfcbf8d713474a35b95be7d3b04adf3e 2023-02-11 00:55:48,680 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/65254bf47f1e46cb9650b4a5194f233f as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f 2023-02-11 00:55:48,683 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:55:48,683 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/bfcbf8d713474a35b95be7d3b04adf3e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e 2023-02-11 00:55:48,687 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 00:55:48,688 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4748df108f96490e961347f1b71a1ed in 33ms, sequenceid=58, compaction requested=true 2023-02-11 00:55:48,688 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4748df108f96490e961347f1b71a1ed: 2023-02-11 00:55:48,688 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,688 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,691 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,691 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 00:55:48,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@db97003[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:48,693 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 00:55:48,693 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:48,695 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:48,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,695 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:48,696 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,696 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,696 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,697 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,697 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,697 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,697 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,697 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,697 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,697 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:55:48,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 00:55:48,701 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 00:55:48,701 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:48,701 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:48,701 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 00:55:48,701 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:48,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,703 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,704 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,704 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,704 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,704 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,705 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,706 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,707 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,707 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,710 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 00:55:48,710 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,710 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,710 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,711 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,711 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,711 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,711 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,711 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,711 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,811 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:48,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:48,822 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 00:55:48,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 00:55:48,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 00:55:48,823 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 00:55:48,823 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,823 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 00:55:48,823 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 00:55:48,824 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,824 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,825 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,825 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,825 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,825 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,826 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,826 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 00:55:48,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,827 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,827 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,827 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 00:55:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41445,1676076840467' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,828 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 00:55:48,828 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 00:55:48,828 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 00:55:48,828 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 00:55:48,828 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,828 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. started... 2023-02-11 00:55:48,828 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 00:55:48,828 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 00:55:48,828 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 00:55:48,828 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,828 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e4748df108f96490e961347f1b71a1ed 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 00:55:48,828 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,38317,1676076840434' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,829 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,830 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,830 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,830 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,831 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,831 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,836 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/9af4aa32129148f08ffb461847581425 2023-02-11 00:55:48,847 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/374ed65cec304b3ea1cdb5948c70318e 2023-02-11 00:55:48,851 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/9af4aa32129148f08ffb461847581425 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425 2023-02-11 00:55:48,854 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:55:48,854 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/374ed65cec304b3ea1cdb5948c70318e as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e 2023-02-11 00:55:48,857 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 00:55:48,858 INFO [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e4748df108f96490e961347f1b71a1ed in 29ms, sequenceid=71, compaction requested=true 2023-02-11 00:55:48,858 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e4748df108f96490e961347f1b71a1ed: 2023-02-11 00:55:48,858 DEBUG [rs(jenkins-hbase4.apache.org,33641,1676076840386)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 00:55:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 00:55:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 00:55:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 00:55:48,859 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,859 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,860 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,860 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 00:55:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33641,1676076840386' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 00:55:48,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44a2105a[Count = 0] remaining members to acquire global barrier 2023-02-11 00:55:48,861 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 00:55:48,861 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,863 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,863 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 00:55:48,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,38317,1676076840434' in zk 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41445,1676076840467' in zk 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 00:55:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33641,1676076840386' in zk 2023-02-11 00:55:48,867 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,867 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,867 DEBUG [member: 'jenkins-hbase4.apache.org,38317,1676076840434' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,868 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,41445,1676076840467' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 00:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,33641,1676076840386' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 00:55:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33641,1676076840386': 2023-02-11 00:55:48,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33641,1676076840386' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 00:55:48,871 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 00:55:48,871 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 00:55:48,871 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 00:55:48,871 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 00:55:48,871 INFO [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 00:55:48,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 00:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 00:55:48,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 00:55:48,872 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,872 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,873 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,873 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 00:55:48,875 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 00:55:48,875 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,875 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,877 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,877 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,877 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,877 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,877 DEBUG [(jenkins-hbase4.apache.org,46819,1676076839233)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 00:55:48,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,877 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,877 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 00:55:48,877 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 00:55:48,877 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,877 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,877 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 00:55:48,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 00:55:48,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 00:55:48,878 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 00:55:48,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 00:55:48,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 00:55:48,978 DEBUG [Listener at localhost/35701] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 00:55:48,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 00:55:48,985 DEBUG [Listener at localhost/35701] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,986 DEBUG [Listener at localhost/35701] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e4748df108f96490e961347f1b71a1ed, daughterA=91fb2cb8d1c0d4219d3c8fa6488ea0bd, daughterB=8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:48,987 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e4748df108f96490e961347f1b71a1ed, daughterA=91fb2cb8d1c0d4219d3c8fa6488ea0bd, daughterB=8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:48,987 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e4748df108f96490e961347f1b71a1ed, daughterA=91fb2cb8d1c0d4219d3c8fa6488ea0bd, daughterB=8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:48,987 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e4748df108f96490e961347f1b71a1ed, daughterA=91fb2cb8d1c0d4219d3c8fa6488ea0bd, daughterB=8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:48,988 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e4748df108f96490e961347f1b71a1ed, UNASSIGN}] 2023-02-11 00:55:48,989 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e4748df108f96490e961347f1b71a1ed, UNASSIGN 2023-02-11 00:55:48,990 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e4748df108f96490e961347f1b71a1ed, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:48,990 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076948989"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076948989"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076948989"}]},"ts":"1676076948989"} 2023-02-11 00:55:48,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure e4748df108f96490e961347f1b71a1ed, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:55:49,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e4748df108f96490e961347f1b71a1ed, disabling compactions & flushes 2023-02-11 00:55:49,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:49,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:49,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. after waiting 0 ms 2023-02-11 00:55:49,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:49,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing e4748df108f96490e961347f1b71a1ed 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 00:55:49,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/dfeedfcf01a14091ab6bd564f01e331d 2023-02-11 00:55:49,161 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/54a9778da9a94c9ca5c24363bb34be9c 2023-02-11 00:55:49,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf1/dfeedfcf01a14091ab6bd564f01e331d as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d 2023-02-11 00:55:49,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:55:49,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/.tmp/cf2/54a9778da9a94c9ca5c24363bb34be9c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c 2023-02-11 00:55:49,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 00:55:49,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e4748df108f96490e961347f1b71a1ed in 29ms, sequenceid=82, compaction requested=true 2023-02-11 00:55:49,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 00:55:49,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:49,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. 2023-02-11 00:55:49,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e4748df108f96490e961347f1b71a1ed: Waiting for close lock at 1676076949141Running coprocessor pre-close hooks at 1676076949141Disabling compacts and flushes for region at 1676076949141Disabling writes for close at 1676076949141Obtaining lock to block concurrent updates at 1676076949141Preparing flush snapshotting stores in e4748df108f96490e961347f1b71a1ed at 1676076949141Finished memstore snapshotting testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676076949142 (+1 ms)Flushing stores of testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. at 1676076949142Flushing e4748df108f96490e961347f1b71a1ed/cf1: creating writer at 1676076949143 (+1 ms)Flushing e4748df108f96490e961347f1b71a1ed/cf1: appending metadata at 1676076949145 (+2 ms)Flushing e4748df108f96490e961347f1b71a1ed/cf1: closing flushed file at 1676076949145Flushing e4748df108f96490e961347f1b71a1ed/cf2: creating writer at 1676076949154 (+9 ms)Flushing e4748df108f96490e961347f1b71a1ed/cf2: appending metadata at 1676076949156 (+2 ms)Flushing e4748df108f96490e961347f1b71a1ed/cf2: closing flushed file at 1676076949156Flushing e4748df108f96490e961347f1b71a1ed/cf1: reopening flushed file at 1676076949164 (+8 ms)Flushing e4748df108f96490e961347f1b71a1ed/cf2: reopening flushed file at 1676076949167 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e4748df108f96490e961347f1b71a1ed in 29ms, sequenceid=82, compaction requested=true at 1676076949170 (+3 ms)Writing region close event to WAL at 1676076949172 (+2 ms)Running coprocessor post-close hooks at 1676076949175 (+3 ms)Closed at 1676076949175 2023-02-11 00:55:49,176 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,176 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e4748df108f96490e961347f1b71a1ed, regionState=CLOSED 2023-02-11 00:55:49,176 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076949176"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076949176"}]},"ts":"1676076949176"} 2023-02-11 00:55:49,178 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-11 00:55:49,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure e4748df108f96490e961347f1b71a1ed, server=jenkins-hbase4.apache.org,33641,1676076840386 in 186 msec 2023-02-11 00:55:49,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-11 00:55:49,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e4748df108f96490e961347f1b71a1ed, UNASSIGN in 191 msec 2023-02-11 00:55:49,187 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=e4748df108f96490e961347f1b71a1ed, threads=14 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,188 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,189 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,189 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,189 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,189 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,271 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,283 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,284 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,310 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,310 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,328 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,340 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,349 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,350 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,350 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,351 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,352 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,352 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3 for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,353 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf for region: e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:49,353 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region e4748df108f96490e961347f1b71a1ed Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 00:55:49,389 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:55:49,390 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 00:55:49,392 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676076949392"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676076949392"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676076949392"}]},"ts":"1676076949392"} 2023-02-11 00:55:49,392 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076949392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076949392"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076949392"}]},"ts":"1676076949392"} 2023-02-11 00:55:49,392 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076949392"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076949392"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076949392"}]},"ts":"1676076949392"} 2023-02-11 00:55:49,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=91fb2cb8d1c0d4219d3c8fa6488ea0bd, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=8921c5dbcb81a1c0de243f8e74e5c732, ASSIGN}] 2023-02-11 00:55:49,395 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=8921c5dbcb81a1c0de243f8e74e5c732, ASSIGN 2023-02-11 00:55:49,395 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=8921c5dbcb81a1c0de243f8e74e5c732, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33641,1676076840386; forceNewPlan=false, retain=false 2023-02-11 00:55:49,396 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=91fb2cb8d1c0d4219d3c8fa6488ea0bd, ASSIGN 2023-02-11 00:55:49,396 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=91fb2cb8d1c0d4219d3c8fa6488ea0bd, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33641,1676076840386; forceNewPlan=false, retain=false 2023-02-11 00:55:49,546 INFO [jenkins-hbase4:46819] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 00:55:49,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=91fb2cb8d1c0d4219d3c8fa6488ea0bd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:49,547 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076949547"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076949547"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076949547"}]},"ts":"1676076949547"} 2023-02-11 00:55:49,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 91fb2cb8d1c0d4219d3c8fa6488ea0bd, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:55:49,550 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=8921c5dbcb81a1c0de243f8e74e5c732, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:49,550 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076949550"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076949550"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076949550"}]},"ts":"1676076949550"} 2023-02-11 00:55:49,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 8921c5dbcb81a1c0de243f8e74e5c732, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:55:49,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:49,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 91fb2cb8d1c0d4219d3c8fa6488ea0bd, NAME => 'testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 00:55:49,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:49,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:49,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:49,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:49,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:49,704 INFO [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:49,704 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1 2023-02-11 00:55:49,704 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1 2023-02-11 00:55:49,705 INFO [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 91fb2cb8d1c0d4219d3c8fa6488ea0bd columnFamilyName cf1 2023-02-11 00:55:49,717 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4-bottom 2023-02-11 00:55:49,720 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f-bottom 2023-02-11 00:55:49,723 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf-bottom 2023-02-11 00:55:49,725 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076-bottom 2023-02-11 00:55:49,728 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425-bottom 2023-02-11 00:55:49,732 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f-bottom 2023-02-11 00:55:49,735 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d-bottom 2023-02-11 00:55:49,736 INFO [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(310): Store=91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:49,736 INFO [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:49,736 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2 2023-02-11 00:55:49,736 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2 2023-02-11 00:55:49,737 INFO [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 91fb2cb8d1c0d4219d3c8fa6488ea0bd columnFamilyName cf2 2023-02-11 00:55:49,747 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d-bottom 2023-02-11 00:55:49,750 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e-bottom 2023-02-11 00:55:49,753 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3-bottom 2023-02-11 00:55:49,756 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5-bottom 2023-02-11 00:55:49,759 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c-bottom 2023-02-11 00:55:49,763 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e-bottom 2023-02-11 00:55:49,766 DEBUG [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb-bottom 2023-02-11 00:55:49,766 INFO [StoreOpener-91fb2cb8d1c0d4219d3c8fa6488ea0bd-1] regionserver.HStore(310): Store=91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:49,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:49,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:49,769 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:49,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:49,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 91fb2cb8d1c0d4219d3c8fa6488ea0bd; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74785033, jitterRate=0.11438383162021637}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:49,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 91fb2cb8d1c0d4219d3c8fa6488ea0bd: Running coprocessor pre-open hook at 1676076949703Writing region info on filesystem at 1676076949703Initializing all the Stores at 1676076949703Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076949703Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076949703Cleaning up temporary data from old regions at 1676076949768 (+65 ms)Cleaning up detritus from prior splits at 1676076949768Running coprocessor post-open hooks at 1676076949770 (+2 ms)Region opened successfully at 1676076949770 2023-02-11 00:55:49,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd., pid=149, masterSystemTime=1676076949700 2023-02-11 00:55:49,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:49,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:49,772 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:49,772 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:49,772 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:49,772 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1912): 91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2 is initiating minor compaction (all files) 2023-02-11 00:55:49,772 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:49,772 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1912): 91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1 is initiating minor compaction (all files) 2023-02-11 00:55:49,772 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2 in testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:49,772 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1 in testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:49,772 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/.tmp, totalSize=34.9 K 2023-02-11 00:55:49,772 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d-bottom] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/.tmp, totalSize=34.9 K 2023-02-11 00:55:49,773 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076947958 2023-02-11 00:55:49,773 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676076947958 2023-02-11 00:55:49,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:49,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:49,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:49,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8921c5dbcb81a1c0de243f8e74e5c732, NAME => 'testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 00:55:49,773 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076948124 2023-02-11 00:55:49,774 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=91fb2cb8d1c0d4219d3c8fa6488ea0bd, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:49,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 00:55:49,774 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076949774"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076949774"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076949774"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076949774"}]},"ts":"1676076949774"} 2023-02-11 00:55:49,774 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676076948124 2023-02-11 00:55:49,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:49,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 00:55:49,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:49,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:49,774 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076948295 2023-02-11 00:55:49,775 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676076948295 2023-02-11 00:55:49,775 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076948460 2023-02-11 00:55:49,775 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676076948460 2023-02-11 00:55:49,775 INFO [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:49,775 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076948639 2023-02-11 00:55:49,775 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676076948639 2023-02-11 00:55:49,776 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076948812 2023-02-11 00:55:49,776 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676076948812 2023-02-11 00:55:49,776 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1 2023-02-11 00:55:49,776 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1 2023-02-11 00:55:49,776 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076948980 2023-02-11 00:55:49,776 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676076948980 2023-02-11 00:55:49,777 INFO [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8921c5dbcb81a1c0de243f8e74e5c732 columnFamilyName cf1 2023-02-11 00:55:49,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-11 00:55:49,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 91fb2cb8d1c0d4219d3c8fa6488ea0bd, server=jenkins-hbase4.apache.org,33641,1676076840386 in 227 msec 2023-02-11 00:55:49,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=91fb2cb8d1c0d4219d3c8fa6488ea0bd, ASSIGN in 384 msec 2023-02-11 00:55:49,787 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] throttle.PressureAwareThroughputController(145): 91fb2cb8d1c0d4219d3c8fa6488ea0bd#cf2#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:49,787 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] throttle.PressureAwareThroughputController(145): 91fb2cb8d1c0d4219d3c8fa6488ea0bd#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:49,789 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4-top 2023-02-11 00:55:49,792 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f-top 2023-02-11 00:55:49,796 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf-top 2023-02-11 00:55:49,799 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076-top 2023-02-11 00:55:49,802 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425-top 2023-02-11 00:55:49,806 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f-top 2023-02-11 00:55:49,807 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/.tmp/cf1/dd688381d70749deab008277c275e121 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/dd688381d70749deab008277c275e121 2023-02-11 00:55:49,808 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d-top 2023-02-11 00:55:49,809 INFO [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(310): Store=8921c5dbcb81a1c0de243f8e74e5c732/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:49,809 INFO [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:49,810 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2 2023-02-11 00:55:49,810 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2 2023-02-11 00:55:49,810 INFO [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8921c5dbcb81a1c0de243f8e74e5c732 columnFamilyName cf2 2023-02-11 00:55:49,811 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/.tmp/cf2/55dd549481c24c699992ab51e15c36ac as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/55dd549481c24c699992ab51e15c36ac 2023-02-11 00:55:49,812 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1 of 91fb2cb8d1c0d4219d3c8fa6488ea0bd into dd688381d70749deab008277c275e121(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:49,812 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 91fb2cb8d1c0d4219d3c8fa6488ea0bd: 2023-02-11 00:55:49,812 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd., storeName=91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1, priority=9, startTime=1676076949771; duration=0sec 2023-02-11 00:55:49,812 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:49,815 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2 of 91fb2cb8d1c0d4219d3c8fa6488ea0bd into 55dd549481c24c699992ab51e15c36ac(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:49,815 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 91fb2cb8d1c0d4219d3c8fa6488ea0bd: 2023-02-11 00:55:49,815 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd., storeName=91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2, priority=9, startTime=1676076949772; duration=0sec 2023-02-11 00:55:49,816 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:49,820 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d-top 2023-02-11 00:55:49,822 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e-top 2023-02-11 00:55:49,825 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3-top 2023-02-11 00:55:49,827 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5-top 2023-02-11 00:55:49,830 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c-top 2023-02-11 00:55:49,832 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e-top 2023-02-11 00:55:49,834 DEBUG [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(539): loaded hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb-top 2023-02-11 00:55:49,835 INFO [StoreOpener-8921c5dbcb81a1c0de243f8e74e5c732-1] regionserver.HStore(310): Store=8921c5dbcb81a1c0de243f8e74e5c732/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 00:55:49,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:49,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:49,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 00:55:49,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:49,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8921c5dbcb81a1c0de243f8e74e5c732; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63473774, jitterRate=-0.05416706204414368}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 00:55:49,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8921c5dbcb81a1c0de243f8e74e5c732: Running coprocessor pre-open hook at 1676076949774Writing region info on filesystem at 1676076949774Initializing all the Stores at 1676076949775 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076949775Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676076949775Cleaning up temporary data from old regions at 1676076949837 (+62 ms)Cleaning up detritus from prior splits at 1676076949837Running coprocessor post-open hooks at 1676076949838 (+1 ms)Region opened successfully at 1676076949838 2023-02-11 00:55:49,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732., pid=150, masterSystemTime=1676076949700 2023-02-11 00:55:49,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:49,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:49,840 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:49,840 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 00:55:49,840 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:49,840 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1912): 8921c5dbcb81a1c0de243f8e74e5c732/cf1 is initiating minor compaction (all files) 2023-02-11 00:55:49,840 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:49,840 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1912): 8921c5dbcb81a1c0de243f8e74e5c732/cf2 is initiating minor compaction (all files) 2023-02-11 00:55:49,840 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8921c5dbcb81a1c0de243f8e74e5c732/cf1 in testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:49,840 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8921c5dbcb81a1c0de243f8e74e5c732/cf2 in testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:49,841 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c-top] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/.tmp, totalSize=34.9 K 2023-02-11 00:55:49,841 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d-top] into tmpdir=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/.tmp, totalSize=34.9 K 2023-02-11 00:55:49,841 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076947958 2023-02-11 00:55:49,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:49,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:49,841 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676076947958 2023-02-11 00:55:49,841 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076948124 2023-02-11 00:55:49,842 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=8921c5dbcb81a1c0de243f8e74e5c732, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:49,842 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676076948124 2023-02-11 00:55:49,842 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076949841"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676076949841"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676076949841"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676076949841"}]},"ts":"1676076949841"} 2023-02-11 00:55:49,842 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076948295 2023-02-11 00:55:49,842 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676076948295 2023-02-11 00:55:49,842 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676076948460 2023-02-11 00:55:49,843 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676076948460 2023-02-11 00:55:49,843 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676076948639 2023-02-11 00:55:49,843 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676076948639 2023-02-11 00:55:49,843 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676076948812 2023-02-11 00:55:49,843 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting 9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676076948812 2023-02-11 00:55:49,843 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] compactions.Compactor(207): Compacting 54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076948980 2023-02-11 00:55:49,844 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] compactions.Compactor(207): Compacting dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676076948980 2023-02-11 00:55:49,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-11 00:55:49,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 8921c5dbcb81a1c0de243f8e74e5c732, server=jenkins-hbase4.apache.org,33641,1676076840386 in 292 msec 2023-02-11 00:55:49,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-11 00:55:49,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=8921c5dbcb81a1c0de243f8e74e5c732, ASSIGN in 451 msec 2023-02-11 00:55:49,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=e4748df108f96490e961347f1b71a1ed, daughterA=91fb2cb8d1c0d4219d3c8fa6488ea0bd, daughterB=8921c5dbcb81a1c0de243f8e74e5c732 in 862 msec 2023-02-11 00:55:49,853 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] throttle.PressureAwareThroughputController(145): 8921c5dbcb81a1c0de243f8e74e5c732#cf2#compaction#144 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:49,853 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8921c5dbcb81a1c0de243f8e74e5c732#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 00:55:49,865 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/.tmp/cf1/2ba59b1cddfa41869efad36fe7d3c916 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/2ba59b1cddfa41869efad36fe7d3c916 2023-02-11 00:55:49,866 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/.tmp/cf2/8f992ee66d6947a2beed90525b249d61 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/8f992ee66d6947a2beed90525b249d61 2023-02-11 00:55:49,869 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8921c5dbcb81a1c0de243f8e74e5c732/cf1 of 8921c5dbcb81a1c0de243f8e74e5c732 into 2ba59b1cddfa41869efad36fe7d3c916(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:49,869 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8921c5dbcb81a1c0de243f8e74e5c732/cf2 of 8921c5dbcb81a1c0de243f8e74e5c732 into 8f992ee66d6947a2beed90525b249d61(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 00:55:49,869 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8921c5dbcb81a1c0de243f8e74e5c732: 2023-02-11 00:55:49,869 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8921c5dbcb81a1c0de243f8e74e5c732: 2023-02-11 00:55:49,869 INFO [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732., storeName=8921c5dbcb81a1c0de243f8e74e5c732/cf1, priority=9, startTime=1676076949839; duration=0sec 2023-02-11 00:55:49,869 INFO [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732., storeName=8921c5dbcb81a1c0de243f8e74e5c732/cf2, priority=9, startTime=1676076949840; duration=0sec 2023-02-11 00:55:49,869 DEBUG [RS:0;jenkins-hbase4:33641-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:49,869 DEBUG [RS:0;jenkins-hbase4:33641-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 00:55:49,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33641] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:56330 deadline: 1676077009987, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed. is not online on jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:50,097 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 00:55:50,103 INFO [Listener at localhost/35701] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 00:55:50,104 DEBUG [Listener at localhost/35701] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 00:55:50,112 WARN [Listener at localhost/35701] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 00:55:50,113 INFO [Listener at localhost/35701] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-11 00:55:50,113 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-11 00:55:50,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-11 00:55:50,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 00:55:50,116 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076950116"}]},"ts":"1676076950116"} 2023-02-11 00:55:50,117 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-11 00:55:50,125 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-11 00:55:50,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=91fb2cb8d1c0d4219d3c8fa6488ea0bd, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=8921c5dbcb81a1c0de243f8e74e5c732, UNASSIGN}] 2023-02-11 00:55:50,126 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=8921c5dbcb81a1c0de243f8e74e5c732, UNASSIGN 2023-02-11 00:55:50,126 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=8921c5dbcb81a1c0de243f8e74e5c732, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:50,126 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076950126"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076950126"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076950126"}]},"ts":"1676076950126"} 2023-02-11 00:55:50,127 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 8921c5dbcb81a1c0de243f8e74e5c732, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:55:50,128 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=91fb2cb8d1c0d4219d3c8fa6488ea0bd, UNASSIGN 2023-02-11 00:55:50,129 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=91fb2cb8d1c0d4219d3c8fa6488ea0bd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:50,129 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076950129"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676076950129"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676076950129"}]},"ts":"1676076950129"} 2023-02-11 00:55:50,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 91fb2cb8d1c0d4219d3c8fa6488ea0bd, server=jenkins-hbase4.apache.org,33641,1676076840386}] 2023-02-11 00:55:50,217 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 00:55:50,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:50,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8921c5dbcb81a1c0de243f8e74e5c732, disabling compactions & flushes 2023-02-11 00:55:50,279 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:50,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:50,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. after waiting 0 ms 2023-02-11 00:55:50,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:50,279 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d-top] to archive 2023-02-11 00:55:50,280 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:50,281 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,282 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,283 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,285 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,287 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,288 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,289 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,290 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e-top, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c-top] to archive 2023-02-11 00:55:50,291 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:50,292 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,293 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,294 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,295 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,296 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,297 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,298 DEBUG [StoreCloser-testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,301 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:55:50,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:50,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732. 2023-02-11 00:55:50,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8921c5dbcb81a1c0de243f8e74e5c732: Waiting for close lock at 1676076950278Running coprocessor pre-close hooks at 1676076950278Disabling compacts and flushes for region at 1676076950278Disabling writes for close at 1676076950279 (+1 ms)Writing region close event to WAL at 1676076950299 (+20 ms)Running coprocessor post-close hooks at 1676076950302 (+3 ms)Closed at 1676076950302 2023-02-11 00:55:50,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:50,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:50,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 91fb2cb8d1c0d4219d3c8fa6488ea0bd, disabling compactions & flushes 2023-02-11 00:55:50,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:50,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:50,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. after waiting 0 ms 2023-02-11 00:55:50,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:50,304 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=8921c5dbcb81a1c0de243f8e74e5c732, regionState=CLOSED 2023-02-11 00:55:50,304 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076950304"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076950304"}]},"ts":"1676076950304"} 2023-02-11 00:55:50,330 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d-bottom] to archive 2023-02-11 00:55:50,331 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:50,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-11 00:55:50,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 8921c5dbcb81a1c0de243f8e74e5c732, server=jenkins-hbase4.apache.org,33641,1676076840386 in 203 msec 2023-02-11 00:55:50,332 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/706969790a704660903b315c07221fbf.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=8921c5dbcb81a1c0de243f8e74e5c732, UNASSIGN in 207 msec 2023-02-11 00:55:50,333 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/886d54b9c7d94704bb768b141a9fa076.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,334 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/c6c000de86b24b5ebdafb3d4d4ef980f.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,335 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/54ba7f21176849db98b8b14273a070d4.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,336 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/65254bf47f1e46cb9650b4a5194f233f.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,337 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/9af4aa32129148f08ffb461847581425.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,338 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/dfeedfcf01a14091ab6bd564f01e331d.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,339 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e-bottom, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed->hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c-bottom] to archive 2023-02-11 00:55:50,340 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 00:55:50,341 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4a78315cfa6d462786efac5e9bc967b3.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,341 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/0c41a68b7bba4020abcb30eeb38b8b9d.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,342 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/f37e6e300c4e4304b11c15d257761edb.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,343 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/4ae7ef72c25747cabcf910bbcb5835e5.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,344 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/bfcbf8d713474a35b95be7d3b04adf3e.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,344 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/374ed65cec304b3ea1cdb5948c70318e.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,345 DEBUG [StoreCloser-testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/54a9778da9a94c9ca5c24363bb34be9c.e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 00:55:50,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:50,348 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd. 2023-02-11 00:55:50,348 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 91fb2cb8d1c0d4219d3c8fa6488ea0bd: Waiting for close lock at 1676076950303Running coprocessor pre-close hooks at 1676076950303Disabling compacts and flushes for region at 1676076950303Disabling writes for close at 1676076950304 (+1 ms)Writing region close event to WAL at 1676076950346 (+42 ms)Running coprocessor post-close hooks at 1676076950348 (+2 ms)Closed at 1676076950348 2023-02-11 00:55:50,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:50,350 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=91fb2cb8d1c0d4219d3c8fa6488ea0bd, regionState=CLOSED 2023-02-11 00:55:50,350 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676076950350"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676076950350"}]},"ts":"1676076950350"} 2023-02-11 00:55:50,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-11 00:55:50,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 91fb2cb8d1c0d4219d3c8fa6488ea0bd, server=jenkins-hbase4.apache.org,33641,1676076840386 in 221 msec 2023-02-11 00:55:50,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-11 00:55:50,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=91fb2cb8d1c0d4219d3c8fa6488ea0bd, UNASSIGN in 227 msec 2023-02-11 00:55:50,353 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676076950353"}]},"ts":"1676076950353"} 2023-02-11 00:55:50,354 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-11 00:55:50,355 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-11 00:55:50,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 242 msec 2023-02-11 00:55:50,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 00:55:50,418 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-11 00:55:50,418 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-11 00:55:50,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 00:55:50,420 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 00:55:50,421 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 00:55:50,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-11 00:55:50,424 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,424 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:50,424 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:50,426 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/recovered.edits] 2023-02-11 00:55:50,426 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/recovered.edits] 2023-02-11 00:55:50,426 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2, FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/recovered.edits] 2023-02-11 00:55:50,429 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/dd688381d70749deab008277c275e121 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf1/dd688381d70749deab008277c275e121 2023-02-11 00:55:50,429 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/2ba59b1cddfa41869efad36fe7d3c916 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf1/2ba59b1cddfa41869efad36fe7d3c916 2023-02-11 00:55:50,429 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/54ba7f21176849db98b8b14273a070d4 2023-02-11 00:55:50,430 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/65254bf47f1e46cb9650b4a5194f233f 2023-02-11 00:55:50,431 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/55dd549481c24c699992ab51e15c36ac to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/cf2/55dd549481c24c699992ab51e15c36ac 2023-02-11 00:55:50,431 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/8f992ee66d6947a2beed90525b249d61 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/cf2/8f992ee66d6947a2beed90525b249d61 2023-02-11 00:55:50,432 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/706969790a704660903b315c07221fbf 2023-02-11 00:55:50,433 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/886d54b9c7d94704bb768b141a9fa076 2023-02-11 00:55:50,433 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd/recovered.edits/90.seqid 2023-02-11 00:55:50,434 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/recovered.edits/90.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732/recovered.edits/90.seqid 2023-02-11 00:55:50,434 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/91fb2cb8d1c0d4219d3c8fa6488ea0bd 2023-02-11 00:55:50,434 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/9af4aa32129148f08ffb461847581425 2023-02-11 00:55:50,434 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/8921c5dbcb81a1c0de243f8e74e5c732 2023-02-11 00:55:50,435 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/c6c000de86b24b5ebdafb3d4d4ef980f 2023-02-11 00:55:50,436 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf1/dfeedfcf01a14091ab6bd564f01e331d 2023-02-11 00:55:50,438 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/0c41a68b7bba4020abcb30eeb38b8b9d 2023-02-11 00:55:50,439 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/374ed65cec304b3ea1cdb5948c70318e 2023-02-11 00:55:50,440 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4a78315cfa6d462786efac5e9bc967b3 2023-02-11 00:55:50,441 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5 to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/4ae7ef72c25747cabcf910bbcb5835e5 2023-02-11 00:55:50,442 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/54a9778da9a94c9ca5c24363bb34be9c 2023-02-11 00:55:50,442 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/bfcbf8d713474a35b95be7d3b04adf3e 2023-02-11 00:55:50,443 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/cf2/f37e6e300c4e4304b11c15d257761edb 2023-02-11 00:55:50,445 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/recovered.edits/85.seqid to hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/archive/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed/recovered.edits/85.seqid 2023-02-11 00:55:50,445 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/.tmp/data/default/testSplitTableRegion/e4748df108f96490e961347f1b71a1ed 2023-02-11 00:55:50,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-11 00:55:50,447 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 00:55:50,448 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-11 00:55:50,449 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-11 00:55:50,450 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 00:55:50,450 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-11 00:55:50,450 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076950450"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:50,450 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076950450"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:50,450 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676076950450"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:50,452 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 00:55:50,452 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e4748df108f96490e961347f1b71a1ed, NAME => 'testSplitTableRegion,,1676076947455.e4748df108f96490e961347f1b71a1ed.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 91fb2cb8d1c0d4219d3c8fa6488ea0bd, NAME => 'testSplitTableRegion,,1676076948985.91fb2cb8d1c0d4219d3c8fa6488ea0bd.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 8921c5dbcb81a1c0de243f8e74e5c732, NAME => 'testSplitTableRegion,41,1676076948985.8921c5dbcb81a1c0de243f8e74e5c732.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 00:55:50,452 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-11 00:55:50,452 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676076950452"}]},"ts":"9223372036854775807"} 2023-02-11 00:55:50,453 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-11 00:55:50,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 00:55:50,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 37 msec 2023-02-11 00:55:50,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46819] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-11 00:55:50,522 INFO [Listener at localhost/35701] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-11 00:55:50,535 INFO [Listener at localhost/35701] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=379 (was 375) - Thread LEAK? -, OpenFileDescriptor=816 (was 835), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=95 (was 103), ProcessCount=168 (was 168), AvailableMemoryMB=2879 (was 2933) 2023-02-11 00:55:50,536 INFO [Listener at localhost/35701] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-11 00:55:50,536 INFO [Listener at localhost/35701] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 00:55:50,539 DEBUG [Listener at localhost/35701] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c8d5aec to 127.0.0.1:56062 2023-02-11 00:55:50,539 DEBUG [Listener at localhost/35701] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,540 DEBUG [Listener at localhost/35701] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-11 00:55:50,540 DEBUG [Listener at localhost/35701] util.JVMClusterUtil(257): Found active master hash=1951274840, stopped=false 2023-02-11 00:55:50,540 INFO [Listener at localhost/35701] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:55:50,542 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 00:55:50,542 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 00:55:50,542 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 00:55:50,542 INFO [Listener at localhost/35701] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 00:55:50,542 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 00:55:50,542 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:55:50,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 00:55:50,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 00:55:50,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 00:55:50,542 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 00:55:50,544 DEBUG [Listener at localhost/35701] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29489d83 to 127.0.0.1:56062 2023-02-11 00:55:50,544 DEBUG [Listener at localhost/35701] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,544 INFO [Listener at localhost/35701] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33641,1676076840386' ***** 2023-02-11 00:55:50,544 INFO [Listener at localhost/35701] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 00:55:50,544 INFO [Listener at localhost/35701] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38317,1676076840434' ***** 2023-02-11 00:55:50,544 INFO [Listener at localhost/35701] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 00:55:50,544 INFO [Listener at localhost/35701] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,41445,1676076840467' ***** 2023-02-11 00:55:50,544 INFO [Listener at localhost/35701] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 00:55:50,545 INFO [RS:2;jenkins-hbase4:41445] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 00:55:50,545 INFO [RS:1;jenkins-hbase4:38317] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 00:55:50,545 INFO [RS:0;jenkins-hbase4:33641] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 00:55:50,545 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 00:55:50,545 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 00:55:50,545 INFO [RS:1;jenkins-hbase4:38317] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 00:55:50,545 INFO [RS:0;jenkins-hbase4:33641] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 00:55:50,545 INFO [RS:1;jenkins-hbase4:38317] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 00:55:50,545 INFO [RS:2;jenkins-hbase4:41445] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 00:55:50,545 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(3304): Received CLOSE for b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:55:50,545 INFO [RS:0;jenkins-hbase4:33641] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 00:55:50,545 INFO [RS:2;jenkins-hbase4:41445] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 00:55:50,546 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:50,546 INFO [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:50,546 DEBUG [RS:0;jenkins-hbase4:33641] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ce197e5 to 127.0.0.1:56062 2023-02-11 00:55:50,546 DEBUG [RS:0;jenkins-hbase4:33641] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,546 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b9396c6e19f6c2ad076604e11fcfef4f, disabling compactions & flushes 2023-02-11 00:55:50,546 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:55:50,546 DEBUG [RS:2;jenkins-hbase4:41445] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6db76ec6 to 127.0.0.1:56062 2023-02-11 00:55:50,546 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:50,546 DEBUG [RS:2;jenkins-hbase4:41445] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,546 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:55:50,546 INFO [RS:0;jenkins-hbase4:33641] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 00:55:50,546 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. after waiting 0 ms 2023-02-11 00:55:50,546 INFO [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,41445,1676076840467; all regions closed. 2023-02-11 00:55:50,546 DEBUG [RS:1;jenkins-hbase4:38317] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e6ed5cc to 127.0.0.1:56062 2023-02-11 00:55:50,546 DEBUG [RS:1;jenkins-hbase4:38317] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,547 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-11 00:55:50,547 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1479): Online Regions={b9396c6e19f6c2ad076604e11fcfef4f=hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f.} 2023-02-11 00:55:50,547 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 00:55:50,547 DEBUG [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1505): Waiting on b9396c6e19f6c2ad076604e11fcfef4f 2023-02-11 00:55:50,546 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:55:50,546 INFO [RS:0;jenkins-hbase4:33641] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 00:55:50,547 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing b9396c6e19f6c2ad076604e11fcfef4f 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-11 00:55:50,547 INFO [RS:0;jenkins-hbase4:33641] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 00:55:50,548 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-11 00:55:50,548 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-11 00:55:50,548 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-11 00:55:50,548 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 00:55:50,548 DEBUG [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 00:55:50,548 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 00:55:50,548 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 00:55:50,548 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 00:55:50,548 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 00:55:50,548 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-11 00:55:50,556 DEBUG [RS:2;jenkins-hbase4:41445] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/oldWALs 2023-02-11 00:55:50,556 INFO [RS:2;jenkins-hbase4:41445] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41445%2C1676076840467:(num 1676076842060) 2023-02-11 00:55:50,556 DEBUG [RS:2;jenkins-hbase4:41445] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,556 INFO [RS:2;jenkins-hbase4:41445] regionserver.LeaseManager(133): Closed leases 2023-02-11 00:55:50,557 INFO [RS:2;jenkins-hbase4:41445] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 00:55:50,557 INFO [RS:2;jenkins-hbase4:41445] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 00:55:50,557 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 00:55:50,557 INFO [RS:2;jenkins-hbase4:41445] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 00:55:50,557 INFO [RS:2;jenkins-hbase4:41445] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 00:55:50,559 INFO [RS:2;jenkins-hbase4:41445] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41445 2023-02-11 00:55:50,565 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 00:55:50,565 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:50,565 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:50,565 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41445,1676076840467 2023-02-11 00:55:50,565 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 00:55:50,565 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 00:55:50,565 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 00:55:50,568 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@dcee87a rejected from java.util.concurrent.ThreadPoolExecutor@9f16d1b[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:50,568 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f/.tmp/info/fb47ad32f57949d2a6c98cd5e5285e66 2023-02-11 00:55:50,569 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41445,1676076840467] 2023-02-11 00:55:50,569 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41445,1676076840467; numProcessing=1 2023-02-11 00:55:50,570 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 00:55:50,572 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41445,1676076840467 already deleted, retry=false 2023-02-11 00:55:50,572 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41445,1676076840467 expired; onlineServers=2 2023-02-11 00:55:50,572 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 00:55:50,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f/.tmp/info/fb47ad32f57949d2a6c98cd5e5285e66 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f/info/fb47ad32f57949d2a6c98cd5e5285e66 2023-02-11 00:55:50,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f/info/fb47ad32f57949d2a6c98cd5e5285e66, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 00:55:50,577 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for b9396c6e19f6c2ad076604e11fcfef4f in 30ms, sequenceid=6, compaction requested=false 2023-02-11 00:55:50,584 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/.tmp/info/4883a6706f0e45fabddad55c65cb9ff4 2023-02-11 00:55:50,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/namespace/b9396c6e19f6c2ad076604e11fcfef4f/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-11 00:55:50,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:50,588 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:55:50,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b9396c6e19f6c2ad076604e11fcfef4f: Waiting for close lock at 1676076950546Running coprocessor pre-close hooks at 1676076950546Disabling compacts and flushes for region at 1676076950546Disabling writes for close at 1676076950546Obtaining lock to block concurrent updates at 1676076950547 (+1 ms)Preparing flush snapshotting stores in b9396c6e19f6c2ad076604e11fcfef4f at 1676076950547Finished memstore snapshotting hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676076950548 (+1 ms)Flushing stores of hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. at 1676076950548Flushing b9396c6e19f6c2ad076604e11fcfef4f/info: creating writer at 1676076950548Flushing b9396c6e19f6c2ad076604e11fcfef4f/info: appending metadata at 1676076950553 (+5 ms)Flushing b9396c6e19f6c2ad076604e11fcfef4f/info: closing flushed file at 1676076950553Flushing b9396c6e19f6c2ad076604e11fcfef4f/info: reopening flushed file at 1676076950573 (+20 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for b9396c6e19f6c2ad076604e11fcfef4f in 30ms, sequenceid=6, compaction requested=false at 1676076950577 (+4 ms)Writing region close event to WAL at 1676076950584 (+7 ms)Running coprocessor post-close hooks at 1676076950587 (+3 ms)Closed at 1676076950588 (+1 ms) 2023-02-11 00:55:50,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676076842496.b9396c6e19f6c2ad076604e11fcfef4f. 2023-02-11 00:55:50,589 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4883a6706f0e45fabddad55c65cb9ff4 2023-02-11 00:55:50,600 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/.tmp/rep_barrier/c4c688fec4754e74a9a73409e433b1ed 2023-02-11 00:55:50,604 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4c688fec4754e74a9a73409e433b1ed 2023-02-11 00:55:50,615 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/.tmp/table/dc61a9ee6ed94d46b3f174be333a118c 2023-02-11 00:55:50,618 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc61a9ee6ed94d46b3f174be333a118c 2023-02-11 00:55:50,619 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/.tmp/info/4883a6706f0e45fabddad55c65cb9ff4 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/info/4883a6706f0e45fabddad55c65cb9ff4 2023-02-11 00:55:50,622 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4883a6706f0e45fabddad55c65cb9ff4 2023-02-11 00:55:50,622 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/info/4883a6706f0e45fabddad55c65cb9ff4, entries=59, sequenceid=210, filesize=11.1 K 2023-02-11 00:55:50,623 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/.tmp/rep_barrier/c4c688fec4754e74a9a73409e433b1ed as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/rep_barrier/c4c688fec4754e74a9a73409e433b1ed 2023-02-11 00:55:50,626 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4c688fec4754e74a9a73409e433b1ed 2023-02-11 00:55:50,626 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/rep_barrier/c4c688fec4754e74a9a73409e433b1ed, entries=24, sequenceid=210, filesize=7.5 K 2023-02-11 00:55:50,628 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/.tmp/table/dc61a9ee6ed94d46b3f174be333a118c as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/table/dc61a9ee6ed94d46b3f174be333a118c 2023-02-11 00:55:50,630 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 00:55:50,631 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc61a9ee6ed94d46b3f174be333a118c 2023-02-11 00:55:50,631 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/table/dc61a9ee6ed94d46b3f174be333a118c, entries=36, sequenceid=210, filesize=8.0 K 2023-02-11 00:55:50,632 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 84ms, sequenceid=210, compaction requested=false 2023-02-11 00:55:50,636 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-11 00:55:50,636 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 00:55:50,637 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 00:55:50,637 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 00:55:50,637 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676076950548Running coprocessor pre-close hooks at 1676076950548Disabling compacts and flushes for region at 1676076950548Disabling writes for close at 1676076950548Obtaining lock to block concurrent updates at 1676076950548Preparing flush snapshotting stores in 1588230740 at 1676076950548Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676076950549 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676076950549Flushing 1588230740/info: creating writer at 1676076950550 (+1 ms)Flushing 1588230740/info: appending metadata at 1676076950556 (+6 ms)Flushing 1588230740/info: closing flushed file at 1676076950556Flushing 1588230740/rep_barrier: creating writer at 1676076950589 (+33 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676076950591 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676076950591Flushing 1588230740/table: creating writer at 1676076950604 (+13 ms)Flushing 1588230740/table: appending metadata at 1676076950606 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676076950606Flushing 1588230740/info: reopening flushed file at 1676076950619 (+13 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676076950624 (+5 ms)Flushing 1588230740/table: reopening flushed file at 1676076950628 (+4 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 84ms, sequenceid=210, compaction requested=false at 1676076950632 (+4 ms)Writing region close event to WAL at 1676076950634 (+2 ms)Running coprocessor post-close hooks at 1676076950636 (+2 ms)Closed at 1676076950637 (+1 ms) 2023-02-11 00:55:50,637 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-11 00:55:50,670 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 00:55:50,670 INFO [RS:2;jenkins-hbase4:41445] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,41445,1676076840467; zookeeper connection closed. 2023-02-11 00:55:50,670 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7028fea rejected from java.util.concurrent.ThreadPoolExecutor@9f16d1b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:50,670 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:41445-0x10040ded7c50003, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 00:55:50,670 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@44dbb6e6] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@44dbb6e6 2023-02-11 00:55:50,670 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7fee1958 rejected from java.util.concurrent.ThreadPoolExecutor@9f16d1b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 311] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:50,747 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38317,1676076840434; all regions closed. 2023-02-11 00:55:50,748 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33641,1676076840386; all regions closed. 2023-02-11 00:55:50,755 DEBUG [RS:1;jenkins-hbase4:38317] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/oldWALs 2023-02-11 00:55:50,755 INFO [RS:1;jenkins-hbase4:38317] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38317%2C1676076840434:(num 1676076842060) 2023-02-11 00:55:50,755 DEBUG [RS:1;jenkins-hbase4:38317] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,755 DEBUG [RS:0;jenkins-hbase4:33641] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/oldWALs 2023-02-11 00:55:50,755 INFO [RS:1;jenkins-hbase4:38317] regionserver.LeaseManager(133): Closed leases 2023-02-11 00:55:50,755 INFO [RS:0;jenkins-hbase4:33641] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33641%2C1676076840386.meta:.meta(num 1676076842208) 2023-02-11 00:55:50,755 INFO [RS:1;jenkins-hbase4:38317] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-11 00:55:50,756 INFO [RS:1;jenkins-hbase4:38317] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 00:55:50,756 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 00:55:50,756 INFO [RS:1;jenkins-hbase4:38317] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 00:55:50,756 INFO [RS:1;jenkins-hbase4:38317] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 00:55:50,756 INFO [RS:1;jenkins-hbase4:38317] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38317 2023-02-11 00:55:50,758 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:50,758 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38317,1676076840434 2023-02-11 00:55:50,758 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 00:55:50,758 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7f90c488 rejected from java.util.concurrent.ThreadPoolExecutor@7baf5fa2[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:50,759 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38317,1676076840434] 2023-02-11 00:55:50,759 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38317,1676076840434; numProcessing=2 2023-02-11 00:55:50,761 DEBUG [RS:0;jenkins-hbase4:33641] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/oldWALs 2023-02-11 00:55:50,761 INFO [RS:0;jenkins-hbase4:33641] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33641%2C1676076840386:(num 1676076842060) 2023-02-11 00:55:50,761 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38317,1676076840434 already deleted, retry=false 2023-02-11 00:55:50,761 DEBUG [RS:0;jenkins-hbase4:33641] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,761 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38317,1676076840434 expired; onlineServers=1 2023-02-11 00:55:50,761 INFO [RS:0;jenkins-hbase4:33641] regionserver.LeaseManager(133): Closed leases 2023-02-11 00:55:50,761 INFO [RS:0;jenkins-hbase4:33641] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 00:55:50,761 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 00:55:50,762 INFO [RS:0;jenkins-hbase4:33641] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33641 2023-02-11 00:55:50,764 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33641,1676076840386 2023-02-11 00:55:50,764 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 00:55:50,764 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@20d08ea0 rejected from java.util.concurrent.ThreadPoolExecutor@24ef1e62[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:50,765 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33641,1676076840386] 2023-02-11 00:55:50,765 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33641,1676076840386; numProcessing=3 2023-02-11 00:55:50,766 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33641,1676076840386 already deleted, retry=false 2023-02-11 00:55:50,766 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33641,1676076840386 expired; onlineServers=0 2023-02-11 00:55:50,766 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,46819,1676076839233' ***** 2023-02-11 00:55:50,766 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-11 00:55:50,766 DEBUG [M:0;jenkins-hbase4:46819] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35731d63, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 00:55:50,766 INFO [M:0;jenkins-hbase4:46819] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,46819,1676076839233 2023-02-11 00:55:50,766 INFO [M:0;jenkins-hbase4:46819] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,46819,1676076839233; all regions closed. 2023-02-11 00:55:50,766 DEBUG [M:0;jenkins-hbase4:46819] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 00:55:50,766 DEBUG [M:0;jenkins-hbase4:46819] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-11 00:55:50,766 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-11 00:55:50,767 DEBUG [M:0;jenkins-hbase4:46819] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-11 00:55:50,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676076841679] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676076841679,5,FailOnTimeoutGroup] 2023-02-11 00:55:50,767 INFO [M:0;jenkins-hbase4:46819] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-11 00:55:50,767 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676076841680] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676076841680,5,FailOnTimeoutGroup] 2023-02-11 00:55:50,767 INFO [M:0;jenkins-hbase4:46819] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-11 00:55:50,767 INFO [M:0;jenkins-hbase4:46819] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-11 00:55:50,767 DEBUG [M:0;jenkins-hbase4:46819] master.HMaster(1502): Stopping service threads 2023-02-11 00:55:50,767 INFO [M:0;jenkins-hbase4:46819] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-11 00:55:50,767 ERROR [M:0;jenkins-hbase4:46819] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-11 00:55:50,767 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 00:55:50,767 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 00:55:50,767 INFO [M:0;jenkins-hbase4:46819] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 00:55:50,767 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-11 00:55:50,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 00:55:50,768 DEBUG [M:0;jenkins-hbase4:46819] zookeeper.ZKUtil(398): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-11 00:55:50,768 WARN [M:0;jenkins-hbase4:46819] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-11 00:55:50,768 INFO [M:0;jenkins-hbase4:46819] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 00:55:50,768 INFO [M:0;jenkins-hbase4:46819] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-11 00:55:50,768 DEBUG [M:0;jenkins-hbase4:46819] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 00:55:50,768 INFO [M:0;jenkins-hbase4:46819] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 00:55:50,768 DEBUG [M:0;jenkins-hbase4:46819] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 00:55:50,768 DEBUG [M:0;jenkins-hbase4:46819] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 00:55:50,768 DEBUG [M:0;jenkins-hbase4:46819] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 00:55:50,768 INFO [M:0;jenkins-hbase4:46819] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.95 KB heapSize=718.97 KB 2023-02-11 00:55:50,781 INFO [M:0;jenkins-hbase4:46819] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.95 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/be3d417d7d424233913eb115bdeefa38 2023-02-11 00:55:50,784 INFO [M:0;jenkins-hbase4:46819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be3d417d7d424233913eb115bdeefa38 2023-02-11 00:55:50,785 DEBUG [M:0;jenkins-hbase4:46819] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/be3d417d7d424233913eb115bdeefa38 as hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/be3d417d7d424233913eb115bdeefa38 2023-02-11 00:55:50,788 INFO [M:0;jenkins-hbase4:46819] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be3d417d7d424233913eb115bdeefa38 2023-02-11 00:55:50,788 INFO [M:0;jenkins-hbase4:46819] regionserver.HStore(1080): Added hdfs://localhost:46775/user/jenkins/test-data/d54dfe53-a951-d206-a0e5-1e5fce1fbd89/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/be3d417d7d424233913eb115bdeefa38, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-11 00:55:50,789 INFO [M:0;jenkins-hbase4:46819] regionserver.HRegion(2947): Finished flush of dataSize ~607.95 KB/622536, heapSize ~718.95 KB/736208, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 21ms, sequenceid=1282, compaction requested=false 2023-02-11 00:55:50,790 INFO [M:0;jenkins-hbase4:46819] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 00:55:50,790 DEBUG [M:0;jenkins-hbase4:46819] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676076950768Disabling compacts and flushes for region at 1676076950768Disabling writes for close at 1676076950768Obtaining lock to block concurrent updates at 1676076950768Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676076950768Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622536, getHeapSize=736208, getOffHeapSize=0, getCellsCount=1499 at 1676076950768Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676076950769 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676076950769Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676076950773 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676076950773Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676076950785 (+12 ms)Finished flush of dataSize ~607.95 KB/622536, heapSize ~718.95 KB/736208, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 21ms, sequenceid=1282, compaction requested=false at 1676076950789 (+4 ms)Writing region close event to WAL at 1676076950790 (+1 ms)Closed at 1676076950790 2023-02-11 00:55:50,794 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 00:55:50,794 INFO [M:0;jenkins-hbase4:46819] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-11 00:55:50,794 INFO [M:0;jenkins-hbase4:46819] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46819 2023-02-11 00:55:50,797 DEBUG [M:0;jenkins-hbase4:46819] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,46819,1676076839233 already deleted, retry=false 2023-02-11 00:55:50,860 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 00:55:50,860 INFO [RS:1;jenkins-hbase4:38317] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38317,1676076840434; zookeeper connection closed. 2023-02-11 00:55:50,860 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@76948c10 rejected from java.util.concurrent.ThreadPoolExecutor@7baf5fa2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:50,860 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:38317-0x10040ded7c50002, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 00:55:50,860 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@19bf6e1e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@19bf6e1e 2023-02-11 00:55:50,861 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@e8aaec8 rejected from java.util.concurrent.ThreadPoolExecutor@7baf5fa2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:50,960 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 00:55:50,960 INFO [M:0;jenkins-hbase4:46819] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,46819,1676076839233; zookeeper connection closed. 2023-02-11 00:55:50,960 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4449f264 rejected from java.util.concurrent.ThreadPoolExecutor@54daef0a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:50,961 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): master:46819-0x10040ded7c50000, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 00:55:50,961 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@47ae433b rejected from java.util.concurrent.ThreadPoolExecutor@54daef0a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:51,061 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 00:55:51,061 INFO [RS:0;jenkins-hbase4:33641] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33641,1676076840386; zookeeper connection closed. 2023-02-11 00:55:51,061 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@74c7c251 rejected from java.util.concurrent.ThreadPoolExecutor@24ef1e62[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:51,061 DEBUG [Listener at localhost/35701-EventThread] zookeeper.ZKWatcher(600): regionserver:33641-0x10040ded7c50001, quorum=127.0.0.1:56062, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 00:55:51,061 ERROR [Listener at localhost/35701-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@c86ff83 rejected from java.util.concurrent.ThreadPoolExecutor@24ef1e62[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 00:55:51,061 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@56cc57a5] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@56cc57a5 2023-02-11 00:55:51,061 INFO [Listener at localhost/35701] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-11 00:55:51,062 WARN [Listener at localhost/35701] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 00:55:51,208 INFO [Listener at localhost/35701] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 00:55:51,241 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 00:55:51,319 WARN [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 00:55:51,319 WARN [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-74375045-172.31.14.131-1676076836114 (Datanode Uuid e08011bb-ec6b-4682-9527-eafae3012242) service to localhost/127.0.0.1:46775 2023-02-11 00:55:51,321 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data5/current/BP-74375045-172.31.14.131-1676076836114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 00:55:51,321 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data6/current/BP-74375045-172.31.14.131-1676076836114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 00:55:51,323 WARN [Listener at localhost/35701] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 00:55:51,325 INFO [Listener at localhost/35701] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 00:55:51,427 WARN [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-74375045-172.31.14.131-1676076836114 (Datanode Uuid 0b950d92-fb4b-4a88-add0-dd4dcb24d701) service to localhost/127.0.0.1:46775 2023-02-11 00:55:51,427 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data3/current/BP-74375045-172.31.14.131-1676076836114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 00:55:51,428 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data4/current/BP-74375045-172.31.14.131-1676076836114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 00:55:51,428 WARN [Listener at localhost/35701] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 00:55:51,430 INFO [Listener at localhost/35701] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 00:55:51,534 WARN [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 00:55:51,534 WARN [BP-74375045-172.31.14.131-1676076836114 heartbeating to localhost/127.0.0.1:46775] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-74375045-172.31.14.131-1676076836114 (Datanode Uuid 37dee797-9923-4df8-b47e-bcee8ee34413) service to localhost/127.0.0.1:46775 2023-02-11 00:55:51,535 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data1/current/BP-74375045-172.31.14.131-1676076836114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 00:55:51,535 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/1bfa8259-8a6f-4741-3932-e66fc38d18b6/cluster_77c88347-c954-8388-6000-cb1a3d5831e6/dfs/data/data2/current/BP-74375045-172.31.14.131-1676076836114] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 00:55:51,558 INFO [Listener at localhost/35701] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 00:55:51,745 INFO [Listener at localhost/35701] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-11 00:55:51,970 INFO [Listener at localhost/35701] hbase.HBaseTestingUtility(1293): Minicluster is down